The speaker treats sleep not as easy rest but as a kind of destination, and a troublesome one. It's compared to "any place to go to": you have to get there, and getting there takes effort. Once inside, it's attractive but messy ("pretty but disordered"), and it holds us longer than we'd like.
The third couplet turns sleep into "a hole full of company." The people or thoughts that fill it "bore and bore," which suggests both tedium and drilling, a repetitive pressure that won't let up. Judging by the imagery, the company could be dreams, memories, or worries that crowd in when the mind should be quiet.
The last lines offer a response: the speaker "lodges" in "my own amber." Amber preserves things, but it also traps them. That image makes the ending feel ambiguous, both a shelter and a stillness that holds the speaker in place. The mood is wry, tired, and a little claustrophobic, like someone who wants rest but can't quite switch off.