Budget onsen towns?

I will be travelling in Japan for 3 weeks next May/June. I'm trying to keep my budget low, but want to experience onsen (I missed it last time I was in Japan).

My planned route at the moment is this:
Arrive at Osaka
First stay: Kyoto
Day trip to Nara and/or Ise
Second stay: Mt Fuji area
Third stay: Tokyo
Finally: Back to Osaka

I'm hoping to make this trip without the rail pass, so I'd rather not travel far and wide. Are there any nice but not so famous (maybe cheaper) onsen cities/towns around this route? I'd prefer a smaller maybe even rural city. Of course the price is not everything, I'd like to have a nice experience (but still stay on budget if possible).

The Mount Fuji area has a wide range of onsen ryokan. Hakone in particular. Most of the places are expensive and many are even overpriced because of Hakone's proximity to Tokyo, but there are some establishments that offer good rates, especially on regular weekday nights.

I'm not sure what sort of cost you're looking at, but we recently visited the Kumano area and stayed the night at Minshuku Atarashiya in Yunomine Onsen. It cost us 15400 in total for the 2 of us, and included 2 delicious meals. Yunomine onsen, Kawayu Onsen and Wataze Onsen are all small places. You could incorporate it into a trip to Ise. Have a look at Nachi Taisha, Hayatama Taisha and Hongu Taisha if you can too. We walked the section of the Kumano Kodo between Yunomine Onsen and Hongu Taisha, which was really serene and beautiful.

Forgot the link http://www.tb-kumano.jp/en/lodging/list.html#yunomine

Thank you for the suggestions!

I checked Hakone but it seems really expensive place. And I'd rather not choose the number 1 tourist spot, if you know what I mean... :) But I'll keep it in mind and try to make our onsen stay on weekdays.