The speaker here wants a kind of ease: to be glad not to wonder, to look at the poor and not really understand them, or at least not feel troubled by the gap. The first lines admit a wish for distance. Seeing "but not discern" suggests a blurred, half-looking kind of attention, the sort we give to things we would rather not think about.
Then the poem turns inward. "Cold justice" in the speaker's breast, set beside "a broken window," is a strange and unsettling pairing. Judging by the images, the broken window might stand for damage, poverty, or a crack in the speaker's own comfort, and the cold feeling in the chest might be a hard, unwelcome sense of fairness or guilt. The poem never says which, and that openness is part of its effect.
The mood is uneasy and quietly self-accusing. It is short and spare, and it leaves the speaker's wish for peace hanging unresolved against a conscience that won't quite go quiet.