The speaker looks back at a past relationship in which someone left, saying they were too young. At the time, the speaker pushed against that excuse and the other person gave way, then walked out. Now a new person has used the same words, and this time the speaker sees what was hiding behind them.
The poem's argument is that "too young" has little to do with age. It really means doubt: the fear of changing your mind, or the sense that the speaker isn't "all you'll ever want and more." The line break after "deep down" and the single word "You know" slow the poem at its most painful moment, as the speaker concludes they weren't the one "made for you."
The mood is quiet and hurt, but also resigned. The speaker says "I don't, really" blame the person for leaving, and the hesitation suggests the forgiveness is still an effort. Repeating "too young" at the end makes it sound like a polite phrase the speaker has decided to accept.