Holding someone's hand is the whole scene here, and the speaker finds a quiet joy in it. The first two lines make the link almost physical: while the hand is held, a smile is held too, as if the smile were something the speaker could carry in the other palm.
The speaker then turns inward. They describe an "inner feeling" and decide to let themselves be happy because of it. That small act of permission suggests happiness isn't something they take for granted. The last line is a little tangled, but its meaning seems clear enough: the speaker wants to give the same happiness to the person they're holding on to. The wording is slightly awkward, and that roughness could be read as shy or sincere rather than polished.
The mood is gentle, intimate and hopeful. Nothing dramatic happens. The poem simply stays inside one tender moment and lets it be enough.