A man heads off to work one morning and never gets there. That's the whole story, and the poem tells it in a few quick lines. The speaker meets a woman who isn't conventionally beautiful, since he says she is "not handsome," but who speaks the best language he has ever heard. He is "taken" with her, a word that suggests being captured as much as being charmed.
What he feels is a mix of delight and helpless surrender. The tone is calm and almost matter-of-fact, which makes the ending funnier and more striking. He "took leave" to go to the office, yet he "could not return." The polite, formal phrasing hides how completely he has been derailed.
The title, "Muse," hints at what is really going on. The woman seems to stand for inspiration, and her words matter more to him than looks or duty. The mood is light, a little wry, and quietly enchanted.