The title sets a sly tone. "Slumming it" usually means someone comfortable pretending to live roughly, and the poem seems to turn that idea around. The speaker belongs to "this end of town," where people "talk highly of hope." The phrasing suggests hope is something discussed, admired and praised rather than actually needed or lived.
Meanwhile, "thousands out of land" have drifted into "the country of want," a neat image that turns poverty into a place people are pushed to. The speaker doesn't shout about this. The tone is flat and observational, almost casual, and that coolness is what stings.
The last stanza brings the unease into focus. "As we increase / and get money which sweetens / all things" sounds content, but the word "sweetens" hints at covering over something bitter. Judging by the tone, the speaker is quietly critical, even self-implicating: "we" profit while others lose ground. The mood is wry, uncomfortable and gently accusing.