The speaker feels caught between two uncomfortable feelings about owning things. At first there's shame: they have too much, and they judge most of it to be worthless, mere "ash." The word suggests something burned out and left over, stuff that has lost its value but still sits in their hands.
Then the poem turns. The speaker admits to regret that there is "no more of it." Even though they look down on their possessions, they miss the abundance, or feel a pang at its running out. That contradiction is the heart of the poem: disdain and longing at the same moment.
The last lines compare this to "sport" that the speaker knows as "hurt." Pleasure and pain blur together, so that enjoying or collecting becomes a way of being wounded. The mood is rueful and self-questioning, spare and a little bitter, with the speaker honestly unable to feel clean about wanting.