Tokyo - Kyoto and in between

I really need some advice for my upcoming trip. We land on Sunday May 4th 2104, 7PM in Narita and will have 6 days in Japan, before we take a cruise from Yokohama on the May 10th at 4PM.

We had plan the following:
May 4th Arrive in Tokyo - Hotel in Tokyo
May 5th Full Day in Tokyo - Hotel in Tokyo
May 6th Mt Fuiji - NOT SURE WHERE - Hotelin the same area.
May 7th - Kyoto, part of day in Kyoto - Hotel in Kyoto
May 8th Full Day in Kyoto - Hotel in Kyoto
May 9th Full Day in Kyoto - Hotel in Kyoto
May 10 - Travel back to Yokohama and have a few hours to see more of Tokyo before the ship leaves.
So basically we end up with 1 full day and a bit in Tokyo, About 2-1/2 days in Kyoto, 1 Full day in Mt. Fuji area.

My questions is, On May 6, it's still the Golden Week holiday. Will I have any difficulty going to visit Mt. Fuji area as far as transportation? I really can't decide where to go to see this, Hakone, Lake Kawaguchiko, or by the Fuji Shibazakura Festival .. or should I just do a day trip and return back to Tokyo?
Are there any Ryokan (2 adults) half decent and our budget is about 130 USD tops (for both of us) in these area?

As far as things to see and do, is my time allocation for kyoto and tokyo reasonable?

May 6 will be very busy in the Mount Fuji area and will see very heavy traffic returning into Tokyo. The opposite direction (e.g. in direction of Kyoto) is not as bad.

If possible, I would avoid visiting Mount Fuji before May 7. And I would also try to avoid traveling from Mount Fuji back into Tokyo on May 6.

If you go to Mount Fuji on the 6th, I would prefer to overnight in the Mount Fuji area that day and travel to Kyoto the next morning when transportation is back to normal.

Thanks for the reply. My fear is that if I book something in Mt Fuji to spend the night on the 6th, is there a possibility I may not be able to get there with the huge amount visitors from Tokyo? If that happens I'll end up with no place to stay.

Say, if I stick to my plan, is there a possibility I won't even be able to get back to Tokyo after my day visit on the 6th? What do you think? I don't mind traffic if I could still get there and get back to Tokyo. Please advise. I'm thinking of going to the
Fuji Shibazakura Festival. Thanks.