The speaker is a diarist, someone who puts things on paper because speaking aloud feels impossible. The opening lines admit this plainly: writing is a substitute for talking, a way to say something without having to say it to anyone.
The middle lines are stranger and more striking. The speaker calls the heart "the devil I pray to," which makes it both a source of temptation and the thing they turn to for guidance. Then comes a blunt thought: nobody else made the heart beat. It is theirs, and it keeps going whether they want it to or not.
The final lines place all of this in "the quick red time / of an evening walk." The phrase suggests sunset, but also a pulse quickened by walking. The mood is quiet, private, and a little uneasy, with a mix of reluctance, devotion, and awareness of being alive.