Most poems of longing want to close a distance. This one wants to open one, and cannot. The speaker is so tightly bound to the "you" he addresses, named in line 2 as "O God", that he has lost the ordinary privacy of being a separate person. By the end he is pleading to be let go: "Devise some means but how I may forsake you."
The first quatrain sets the trap. Even when the speaker is by himself, the "you" is there ("You not alone, when you are still alone"), so there is no corner in which to be "private." The next two lines pivot on a repeated word: "Since you one were, I never since was one; / Since you in me, my self since out of me." "Since" means both "because" and "from the time that," so each line gives a cause and a date at once. "One" is played against itself too. Union with the "you" should make the speaker whole, yet he has never been "one" since it happened. Joining has divided him.
The second quatrain describes the experience itself. Here the rhythm changes. The poem moves in fairly regular iambic pentameter, but all four lines ending "being," "either," "seeing," "together" finish on an unstressed syllable. The lines seem to float off their last beat, just as the speaker feels "Transported from my self." "Senseless with too much joy" works twice: he is overwhelmed, and he is also literally without senses. Then comes the sharpest paradox: "only absent when we are together." Closeness feels like absence, presumably because at the moment of meeting there is no longer a separate self to do the meeting. Distance, oddly, is where both remain "present yet to either."
At line 9 the poem turns and becomes a set of demands: "Give me my self and take your self again." The speaker writes "my self" as two words throughout, so the self becomes a possession that can be handed back and forth. The next lines pursue that idea of ownership with something like a legal argument that backfires. So much of what is his "doth with you remain" that reclaiming it means carrying the beloved away too: "taking what is mine, with me I take you." The vocabulary of returning, taking and owing is cold and practical, which makes the helplessness underneath it more striking. He cannot even end the relationship cleanly, because he cannot sort out whose property is whose.
The couplet gives up on argument and falls back on the oldest explanation available: "You do bewitch me." Magic is what you blame when reason has failed. The last line twists its grammar to match the confusion: "From my self you, or from your own self I." "My self you" is the "you" who has become part of the speaker's self, and "your own self I" is the "I" who has become part of the other's. The pronouns have fused into compound phrases, so fleeing one means fleeing both. The rhyme makes the point as well. "Fly" is answered by "I," and the escape the speaker wants ends on the very self that is already tangled up.
Elsewhere the language is plain. Almost every word is a single syllable, and the hardest words ("private," "transported," "devise") are not obscure. The difficulty lies in the logic, which keeps doubling back on itself. The rhyme scheme is the familiar abab cdcd efef gg, though to a modern ear "alone" and "one," or "either" and "together," land closer to echoes than to exact rhymes. The settled pattern holds an argument that is coming apart.
The poem matters because it reverses a familiar expectation. Declarations of devotion usually celebrate being joined to the beloved, or mourn being kept apart. Here union is the problem, and what is mourned is the loss of a self. Because the address "O God" sits alongside language of enchantment and personal possession, the poem keeps open whether the speaker is overwhelmed by divine presence or by a person, and the experience it describes fits either. What it captures is the vertigo of being known and held so completely that you can no longer say where you end.