The speaker says they can't go to church in the usual sense. The only place they can worship is a closet, a small, private, enclosed space. The title plays on the word "closeted," which suggests hiding or keeping something secret, though the poem never says what might be hidden. That leaves the reading open: it could be doubt, a private kind of faith, or something else the speaker doesn't feel free to share.
The second half turns stranger and more image-driven. "Wing of my evening" suggests something like a bird or a flight of thought rising at dusk, and it goes "up with that cold cold eye." The repeated "cold" makes the gaze feel distant, maybe judging, maybe simply indifferent. I read it as a watchful presence, whether God, the moon, or the speaker's own conscience.
The mood is hushed, tense, and a little lonely. Faith here is intimate and uneasy rather than comforting.