The title is the key: a curator looks after objects in a collection, and here the speaker seems to be both the keeper and one of the exhibits. Up high, "among things I found," he describes himself as someone who has "made myself old," as if age were something assembled on purpose, "a copy of a good original." That's a modest, slightly self-mocking admission. He's an imitation of something better, perhaps an earlier self.
Yet the tone isn't gloomy. "Light-headed and raw" suggests giddiness and exposure, and "profane as a lived-in book" makes wear and mess sound like virtues: a book is worth more once it's been read and handled. The last lines, a "little orange horse / which is old hat," land with a shrug. The object is charming but stale, and the speaker knows it. The mood is wry, tender and playful, a mix of humility and quiet pleasure in being worn.