The speaker looks back on a night with someone and works out what really made them fall for that person. It wasn't the sex, and it wasn't the things said that night, which might be what we'd expect. It was the quiet moments the next morning: the kisses, the cuddling, the held hands.
The final line is what gives those small things their weight. Everyone, or at least the speaker, knew this was the last time and that it "could never be again." So the tenderness carries grief inside it. The poem suggests the speaker loved the person most when the closeness was already ending.
The mood is soft, sad, and honest. The short, plain lines and the repetition of "it wasn't" followed by "it was" feel like someone thinking out loud, correcting themselves until they reach the truth. The lack of dramatic language makes the ache feel more real, as though the speaker is still quietly holding on to that morning.