Shibazakura Festival or Hakone?

I'll stay 5 days in Tokyo but I'm planning to do a 2 days-trip out of the city. I'll be there at the beginnig of May.

My first idea was to travel early in the morning from Shinjuku to Kawaguchiko to see the Fuji and the flowers (Shibazakura festival) and, after lunch, move to Hakone to stay in a ryokan with onsen and kaiseki dinner and traditional japanese breakfast, with the following day used to visit Hakone area.

Now, I have two problems:
1) I sow that the connections with buses between Kawaguchiko and Hakone are not fast and easy (expecially between Gotemba and Gora);
2) the Shibazakura Festival is not in Kawaguchiko and, based on the pictures in the web, that offers the best shoot of Fuji with flowers.
I would like to know if it'll be better to do the two visits in distinct dates like day-trip and give up on the ryokan experience. In this case I will choose the days with the best weather of course.
Otherwise, if I have to choose only one destination I'm very hesitating on which one it's better.
Sorry if I was polix ;)

1. By train, it takes more than 4 hours from Kawaguchiko to Hakone-Yumoto (Hyperdia), so I don't think it is a good idea to reach Hakone from Kawaguchiko.

2. During planning my coming trip in Arp/2015, I was also confused about Shibazakura Festival and Hakone. Finally, I go with Shibazakura Festival because it is best to know the most beautiful icon of Japan - Mt. Fuji. If you check some pictures, Shibazakura Festival is the best to view Mt. Fuji with magnificent flower colour, especially for the first-time visit.

3. You could have a full day in Shibazakura Festival with some more places near Kawaguchiko:
http://www.jreast.co.jp/e/fuji/course/fuji_sample01.html
Afterwards, go back to Tokyo.

You could spend one more day to Hakone from Tokyo, s pend a night there and go back to Tokyo or the next places.