Herbert's speaker begins in a sulk. He is "full of rebellion" and lists the ways he might escape God: he would "die, / Or fight, or travel, or deny" any connection at all. Then, within a few lines, he is on his knees. Herbert was an Anglican priest, so it is no surprise that the poem reads as a prayer, but it is not a polite one. Its central idea is that human nature, left to itself, cannot mend itself. The title is the diagnosis: this is the heart's natural state, and only God can change it. The speaker doesn't pretend to be good. He asks to be overpowered.
The form helps the argument. Each of the three stanzas runs on the same pattern, two rhyming lines, then a third that waits for its partner, then a rhyming pair, then a closing line that answers the third. In the first stanza "me" is answered by "thee," so the human and the divine are literally tied together by rhyme at the stanza's edges. The line lengths do the rest. Three full lines of complaint are followed by a sudden short one, "O tame my heart," just four syllables. The speaker's long, restless sentences are cut off by a plea, and the next line, "It is thy highest art," is also short, as though he is catching his breath. Only the last line of the stanza stretches back out.
The imagery of the first stanza is a little startling. To "tame" a heart is to treat it as a wild animal, but the stanza ends with a military image: God's "highest art" is "to captivate strong holds to thee." A stronghold is a fortress, and the speaker casts his own resistance as a defended castle. The odd flattery is that the harder he is to take, the more glory there is in taking him.
In the second stanza the tone turns darker and almost bargaining. Rebellion becomes "venom" that "lurk[s]" and "fume[s] and work[s]," words that suggest poison, but also something fermenting in a closed vessel. If it is left alone, the soul will "turn to bubbles" and "vanish into a wind." The phrase "by kind" means "by its nature," so the title quietly slips back into the poem at exactly the moment the soul dissolves. The last line is the boldest in the poem: the vanished soul would make God's "workmanship deceit." The speaker half-threatens God with embarrassment. If you let me go to ruin, he says, your handiwork turns out to be a fraud. It is a daring argument to put to your maker, and it shows how close to the surface Herbert's honesty sits.
The final stanza changes the picture from fortress to stone. The heart is "rugged," and the speaker wants God to "smooth" it and "engrave" his law there, like words cut into a tablet. The poem then risks a second, more radical request: "Or make a new one." The old heart is "sapless," a word that belongs to wood and plants rather than rock, so the heart seems both hardened and dried out, a thing that has lost its life. Judging by the imagery, the pun in "stone" does the damage: it is "a much fitter stone / To hide my dust, than thee to hold." The heart has become a gravestone. The word "hold" quietly answers "strong holds" from the first stanza. God was to capture the stronghold, but the heart, as it stands, cannot even contain him.
The poem is significant because it avoids easy piety. It is made of plain, mostly one- and two-syllable words, yet it moves through animals, castles, poison, air and graves without strain. And it gives rebellion a voice before it gives way to obedience. The ending is not a victory. It is a request, still unanswered, from someone who knows exactly how stubborn he is.