In just three lines, the speaker watches his wife walk by in a light silk gown and sees something sacred. He calls her "a little church in the middle of church," a striking image that is easy to read two ways. She might be a small sanctuary inside a larger one, a place of calm within the world's grander ceremonies. Or the speaker may mean that she is, for him, the truest holy place in whatever setting they share. The poem doesn't say where she is, so the reader fills that in.
The opening "O" sounds like a gasp or a quiet prayer, and it tells us how moved he is. The feeling is tenderness mixed with awe. Her gown is "light" and "silk," so she seems delicate and almost weightless. The last word, "walking," stands alone, which slows the poem down and lets us picture her steady movement. The mood is hushed, reverent, and intimate, as if love and worship have become the same thing.