The speaker is a man pulled between two women, and the title, "Torn," sums up his state. He moves "from wife to mistress," and the poem suggests he lives a kind of double life, getting by on "wit," which he calls "gaudy but poor": flashy, quick, but not worth much underneath. There's a note of self-mockery in that phrase.
The most striking moment comes when he says that, meaning to see his wife, "I was not I." He seems to lose himself, as if desire has taken over his identity. He then compares himself to a moth drawn to the sun, an image of someone who knows, or should know, that the attraction will hurt him. Yet he doesn't pull away. He burns, and even takes "pleasure / in the morning pain."
The mood is restless and guilty, but also strangely willing. He feels trapped by his own appetite, and the poem leaves him there, half ashamed and half enjoying it.