The speaker describes a wandering mind, one that has always drifted off the rails and never bothered to come back. Other people called this habit a "tragedy," but the speaker seems content, since the world is bigger than a single route. Then someone new appears, a "little heart thief" the speaker feels they should scorn, and that person becomes the only thing the speaker's thoughts follow.
The central joke and the central feeling are the same: love is a train. The speaker sees a crash coming and argues against it, yet the wheels keep turning and the "hot coals" won't cool. The mood is playful and a little helpless. The sing-song ending, "Chugga-chugga choo-choo!", shows the speaker giving up the fight and enjoying the ride. The rhyming couplet at the close makes the surrender feel cheerful rather than frightening.