How would you be traveling around? For some places, it's easy enough to go by public transport, but for other more out of the way places, rental car is better.
Here are a few more specific suggestions:
Choushi in Chiba:
https://www.city.choshi.chiba.jp/english/guide.htmlCute lighthouse and train, the fish is awesome, home of shoyu so you can do a tour, nice onsen, and scenery, etc.
Kofu area of Yamanashi:
http://www.yamanashi-kankou.jp/foreign/english/area/kofu.htmlAnd if you like trains, the Maglev test track in Yamanashi as well:
http://linear-museum.pref.yamanashi.jp/english/index.htmlKasumigaura in Ibaraki, depending on when in October you could catch the Hobiki boats:
http://english.ibarakiguide.jp/news/sightseeing-boat-hobiki-sen.htmlAlso in Ibaraki, Fukuroda Waterfall and Rokkakudo are in Daigo town, which is a kinda rural area but nice.
Takasaki in Gunma as mentioned a few times, lots of museums and shrines, lots of music, good city to visit for a day or weekend. In the area is the new Unesco World Hertitage Sites like Tomioka Silk Mill or Takayama in Fujioka. Kiryu for old-town feel and kimono/silk. Kusatsu, Ikaho Onsen or Minakami area.
Nearby Ashikaga in Tochigi for the oldest school in Japan, wine vinyards, nice shrines in the area, and Mashiko for pottery, Utsunomiya for gyoza and jazz.
Chichibu area in Saitama.
If you really want out of the way, visit Tokyo's islands, Oshima and Niijima. Famous for kusaya!