Is it even possible to rent an apartment as a 'tourist'? Yes it is possible. I've stayed in an apartment in Tokyo multiple times, pretty much every time I go to Japan. The company I use is
http://www.liveinasia.com/ and they specifically rent apartments to tourists / foreigners staying a short time in Tokyo.
what I'm currently wondering is whether to have a single 'base of operations' somewhere in a geographically central location from which I could travel to all around the islands and then return at the end of the day or the nextI wouldn't recommend that. You would be spending a lot of time just traveling rather than getting to see and know Japan. Plus on multi-day trips you'd be paying twice for accommodation...once at your "home base" and again in the area you are staying overnight.
There are hostels throughout Japan, but I've never stayed in them so I cannot recommend any. But, what I can recommend is a chain of business hotels that may meet your needs and budget.
Toyoko Inn (
http://www.toyoko-inn.com/eng/index.html) has hotels all across Japan, as well as a few in Korea and one in Cambodia now. They are like the McDonald's of hotels...they're all the same...once you've stayed in one you've pretty much stayed in the them all. The rooms all come with wifi and wired internet, a tv, a fridge, a safe, and a kettle. And all Toyoko Inns offer laundry facilities and complimentary breakfast each morning.
And I find them fairly inexpensive, compared to other hotels. You can register as a member which gets you 20% discount on Sundays and Holidays, 5% discount Monday-Saturday, and a free stay after every 10. Once you've earned a free stay you can use it right away or you can hold onto it and use it at a later date. Membership also allows you to book a room 6 months in advance, non-members can only book 3 months in advance. Membership is a one-time only fee of 1500Yen and is definitely worth it, especially for the free nights you earn.
Also, one tid-bit of information. If you use the Toyoko Inn reservation system in English they require a credit card number to reserve the room...but when you check-in you can pay in cash if you'd like. However, if you use the reservation system in Japanese, they don't require a credit card number. I'm not sure why that is but if you have both the English page open and the Japanese page open it is simple to fill out the Japanese reservation page using the English page as a guide.
I've stayed in Toyoko Inns all across Japan (from Hokkaido to Okinawa), and one in Seoul, and never had any issues or problems.