Davis turns a rainy day into a small scene in a music store. The rain becomes a curious old man wandering among the instruments, and the houses become the strings of a harp, some high and some low. The image works because rain really does sound like plucking: drops tapping on roofs, each one a slightly different note.
The old man plays with thin, long fingers, which suggest the thin lines of falling water. What he plays is a "weary jingle" and a "soft jazzy jangle," so the rain sounds tired but also relaxed and bluesy. The speaker seems fond of him, watching with gentle amusement rather than gloom.
The ending adds a little humor. The old man slips away "before the boss comes 'round," like a customer or worker who knows he shouldn't be touching the merchandise. The rain stops, and the music goes with it. The mood is quiet, playful, and a bit wistful.