The speaker is talking to someone who made promises and kept letting them down. The "pinky promise" is a childhood ritual, a small, serious pledge, and the speaker takes it literally: if you break a promise, you should break a pinky. Otherwise, the poem says, a promise is worthless, because there's no price for failing to keep it. The person addressed paid nothing. Worse, they "gained and gained and gained," and the repeated word makes it sound like a pile of profit built on other people's disappointment.
The last two lines turn the poem inward. The speaker says they're "unafraid / of broken pinkies too," which could mean they've been hurt so often that the threat of pain no longer scares them. The tone is bitter but quiet. There's anger here, along with a tired, numbed sort of toughness, and the playful image of a pinky promise makes the hurt feel sharper.