The speaker is repeating what other people said about someone: "Don't trouble yourself over him. He'll be just another book on the shelf." The title, "Books," suggests the poem is built on a metaphor in which a person is compared to a book. To the others, he is ordinary and easy to replace, one among many, and not worth any worry or attention.
The last line turns this around. The speaker quietly says that those people didn't know something: he was their favorite book. That gentle admission tells us the speaker cared deeply, perhaps more than they ever let on. The poem doesn't say what happened to him, so we're left to guess whether he left, was lost, or was simply dismissed.
The mood is wistful and a little sad. The ellipsis before the final line feels like a pause, as if the speaker is hesitating before sharing something private. It's a small, honest moment about how differently people can value the same person.