The speaker is caught in one-sided love. They begin by listing what they wish someone would do for them: remember small details, smile at their good days, feel afraid of losing them. Each wish is measured against what the speaker already does for "you," so the poem quietly shows how much they have been giving without getting anything back.
Then the ending turns the poem around. The speaker decides they don't want anyone to love them this way, because they know how painful it is. The final line is the most striking: if someone looked at them the way they look at "you," that person would see how "selfish" the speaker was. It's hard to tell exactly what the speaker means, but the poem suggests guilt, as if their longing has been a burden on the person they love. The mood is tender, tired and quietly self-critical.