The speaker looks back to being five years old, when the world felt unknown and every bend in the path might hide something. What scared the child most wasn't the neighbor's ghost stories but a woman's quavery voice telling of a headless hunter whose lantern glowed on moonless nights. The poem suggests this woman was an older figure, perhaps a relative or neighbor, and the detail of her wooden legs is oddly vivid. It's unclear whether her voice or her legs frightened the child more, and that blur feels true to how memory works.
The ending shifts to the present. The speaker still looks for her house, but only a red privy is left, with its "vacant hole." The fear has faded, replaced by loss. The mood moves from spooky childhood thrill to quiet, wistful emptiness. The hole becomes a small image of absence: the storyteller, her house, and the child who listened are all gone.