A pianist in a bar warms up the way he always does, with a joke-like origin story that the drinking crowd has heard before. He knows it's corny, so he rushes through it and mocks it, and the audience laughs. But they're mostly there to drown their own sorrows, and they don't feel much sympathy for him. The room is full of what the poem calls "phony joy."
Everything changes when he closes his eyes and plays the opening chord. The smile disappears, the laughter stops, and the music carries the same story he told as a gag, now told honestly. The suggestion is that he is speaking to someone who is no longer there, saying things he never said to her. The listeners feel it, and the mood turns from loud and shallow to quiet and aching. In the final lines they realize his story is partly theirs too, so the sadness becomes something shared.