The poem opens with a small, strange image: a tower that knows nothing of the hill. Each stands in its own place, and neither has any idea what the other is like. That sets up the poem's idea about wandering. To know a place, you have to leave the one you're fixed in.
The second stanza turns travel into "a sort of music," but a fake one, "counterfeiting hands and discourses." Travel seems to imitate human touch and conversation without really being them. The speaker also says this experience is "not fit for paper," so it resists being written down, which is a bit ironic in a poem.
The ending is the most unsettling part. The wanderer is "like a man gone / out of his quiet mind." Wanderlust is compared to a kind of madness, or at least to losing calm. The mood is restless and uneasy, pulled between longing and doubt. The speaker seems drawn to leaving but wary of what it does to the mind.