The poem watches a face change over time, and the title's "facial recognition" suggests the strange work of recognizing someone as they alter. It begins with an invitation to feel how soft a face is, almost like touching a child's cheek or a lover's skin. Then "by and by" the same face becomes hard. The poem doesn't say exactly whose face this is, so it could be another person's, or the speaker's own, seen in a mirror.
The middle lines carry the emotional weight: "you cannot break it / you cannot bear to see it." The hardness seems both stubborn and painful, something you can neither change nor look at for long. The speaker's answer is quiet retreat. Leave it alone, eat dry toast, drink good wine. It reads as a small, modest comfort, a bit rueful and a bit resigned.
The closing repetition, "by and by / how late it is," shifts from the face to time itself. The mood is gentle, wistful, and tired, with a sense that the hour has slipped past without anyone noticing.