The speaker keeps pulling away from someone, and the other person keeps reacting the same way. Sitting apart, looking away, even leaving the room: nothing stops the smile or the blush. At first that seems like proof of affection. Then the poem turns on the line "Maybe it was never me." The speaker realizes the smile may have been meant for someone else all along, and that they simply couldn't see it.
The last three lines hit harder. The speaker says this person was involved with "others," and those others were people the speaker called brothers. That suggests a double sting: a romantic misreading and a sense of betrayal by friends. The final line, "Now I'm just used by another," is blunt and bitter, though the poem leaves unclear exactly how the speaker is being used.
The mood is quiet, hurt, and a little resigned. Short lines and simple rhymes make it sound like someone talking themselves through a painful realization.