The poem is a short, sharp look at what it means to be comfortable while others aren't. It opens with a mole, a creature that burrows out and comes back to its hole, and the speaker seems to see themselves in it: someone who ventures into the world and retreats again, with nothing for company but "an ugly wish." The poem never says what that wish is, which makes it more unsettling. Judging by the title, it may be a wish to stay insulated, or to be rid of guilt.
The image of "hands clean / as hats on Sunday" suggests respectability, even smugness, a person who hasn't had to get dirty. The last lines turn the idea around: the speaker is "at home with the poor / people out." It can be read as being comfortable among the poor, or as feeling at home only once the poor have been sent away. The mood is wry, uneasy, and self-accusing, as if the speaker is quietly judging their own good fortune.