The poem opens with a teasing question: who wouldn't want to be kissed by castrati, those eunuchs with women's voices? The phrasing is playful, but it also hints at something strange and a little unsettling, a beauty that comes from loss and ambiguity.
The middle lines shift into a more general thought: what we already know and understand will never please us as much as what we don't. That seems to explain the speaker's attraction. The mystery of a voice that doesn't match its body is part of the appeal.
The last stanza is a puzzle. "That anagram of dog" is God, and the speaker says the tongue is "more judicious" than him. The tongue could be the singing voice, or the kissing mouth, or just language itself. Either way, the speaker seems to prefer physical, earthly pleasure to divine authority. The mood is witty, sly, and slightly irreverent, with a thread of longing running underneath.