The poem begins in the middle of something, with "they" who seem not to notice a sound. We never learn who "they" are, and that gap is part of the effect. The speaker stands slightly apart, hearing something others either miss or ignore.
The second stanza is a little puzzling. "Did not seem fit for it" could mean the listeners weren't suited to such music, or that the music wasn't suited to them. Either way, there's a mismatch between the sound and the people around it. Then the poem finally names the source: "the great harp / that I hear in the garden / after dark."
The word "great" gives the harp size and wonder, though we can't tell whether it's a real instrument, the wind, or something imagined. The speaker seems quietly moved, maybe a little lonely, but also privileged to be listening. The mood is hushed and mysterious, like a secret shared only with the reader.