Magome-Tsumago hike worth a detour?

I'm not so happy with my journey planning at the moment because there are many days when I've got to get up very early to do all the things I want to do. So I'm just thinking about skipping some attractions.

I've planned to hike the trail between Magome and Tsumago on my way from Tokyo to Kyoto. That means, I would start in Ueno at about 7am, go to Magome by shinkansen, train and bus, arrive there at about 11.30-12.00, hike to Tsumago in about 3 - 4 hours and take bus, train and shinkansen to Kyoto, where I would arrive in the evening.

Do you think all the time for transportation is worth the visit in Magome and Tsumago and the trail? In fact, I thought it is, but I'm not so sure any more. I will also travel to Hakone, to Koyasan, Kanazawa, Shirakawago and Takayama, so I will spend some time out of the big cities.

Now I also think a visit in Hikone on the way might be nice instead and much easier to attain.

You can spend a night in Matsumoto to make it a more relaxed trip. I did it last month. I went from Utsunomiya to Nagano, spent the night there. Next morning I looked around Nagano and took a train to Matsumoto in the afternoon. I spent the night in Matsumoto and early the following morning I too another train to Narai, then a train to Nagiso and then a bus to Tsumago and another to Magome and finally another to Nakatsugawa so I can take a train to Nagoya to finally take a bullet train into Osaka late in the evening, with plenty of time to stuff myself in Dotonbori. I spent about 1.5 hours in each town (Narai, Tsumago and Magome) I'm sure you'd have enough time to walk from Tsumago to Magome if you skip Narai or spend less time in each town. I think the downhill walk would be the Tsumago to Magome route. (At least it seemed to me)

Thank you, that's not a bad idea. But unfortunately spending the night in Matsumoto means being more in a hurry on the day before the hike, and it means adding an additional hotel change to my journey.