I love this poem. It is contemplative; the poet is looking back on his life and seeing the striations laid down by the events of that life, like geological layers.
Looking at the walls of a canyon, the Grand Canyon, for example, one can see those geological layers stacked one on the other, building all the way up to the surface. And so it is with our lives. Some of those layers are ones we would just as soon forget, but they all are a part of what made us who we are.
The poet is saying that we should not be distracted by the litter of our daily lives; we should embrace and live in the layers, the solid stuff that made us.