The speaker looks up at a sky that feels flat and empty, a dull blue fading into a dull orange, and realizes the stars are gone. That loss hurts in two ways. They can no longer trace constellations, and they can no longer hear the comfort of a "you" whose whispers once soothed their restless mind. It is hard to say whether that person has left or whether the stars themselves stood in for their comfort, and the poem leaves that blur in place.
The tone turns from private sorrow to something shared. The speaker says "we" are all aching, as if pain is carved into the bones of anyone who has gone unloved. The ending is the sharpest part: the stars were not lost by accident, "we are the ones who drove away the stars." The mood is weary and resigned, with a quiet guilt. The repeated "ache" makes the sadness feel like a habit the speaker accepts rather than escapes.