The speaker is a gardener who has stopped fussing and simply lets the garden grow. Instead of controlling it, he sings to the dirt, and the poem's pleasure comes from that loose, unforced attitude.
The second stanza turns the song into a kind of toast. He calls the garden "my love" and "my drudging wench," a phrase that mixes affection with a rough, earthy teasing. A wench who drudges is someone who works hard, and the line suggests the garden labors on his behalf, or perhaps that gardening is its own hard, humble work. Then he asks it to "be drunk as me." He wants the garden to share his tipsy joy, as if growth itself were a form of intoxication.
The mood is playful, warm, and a little rowdy. The poem is short and loose, much like the tipsy mood it describes. It treats the garden less as property and more as a companion to celebrate with.