The speaker steps out in a coat and sword, as if marching in a ceremony, with drums sounding all around. The title, "Public servant," suggests this is a official role, and the costume and noise make it feel grand and a little absurd. The crowd watching is called vulgar and is pulling ugly faces, so the speaker seems to feel looked down on, or at least misunderstood, by the people he serves.
The last lines turn inward. The speaker doesn't think the audience knows "the wine," which I read as a private pleasure or secret inner life, a place where he can "kill the clock" that "talked on / about business." Time, work and talk of duty become something he can silence for a moment. The mood is wry and a bit defiant: outward pomp and public scorn on one side, quiet escape on the other. The poem seems to say that a person can perform a role for everyone else while keeping something of himself apart.