The speaker is nursing a broken heart. Love once felt wonderful, but losing it left them "burning" with regret, and they wish they could simply forget it. The rose and thorn image sums up the pain: something beautiful that hurt them when they reached for it. Their heart is "frozen," and even after trying to move on, they still mourn.
Then the poem shifts. Wandering into a "new garden," the speaker tries to numb themselves to all the roses, but a swan draws them in with a lovely song and graceful, teasing poses. Judging by the imagery, the swan stands for a new person or a new chance at love. Even so, the speaker's mind is still unsettled and hurting. The swan's song nudges them to find a way forward, and the final couplet settles on a hopeful note: they could happily stay here. The mood moves from bitter and weary to cautious, quiet relief.