The aquarium is a must see depending on your love of aquariums I suppose. I found it impressive and have visited twice. If you do go, try to get there right when it opens to avoid the crowds that can build up. I think spending the morning there and then going for lunch nearby before heading off somewhere else would be the best bet if you wanted to go to it.
If you want to hit all the spots in Osaka and maybe enjoy a ferris wheel or boat tour as well, I'd suggest using the Osaka pass for one or two days. It gives you unlimited transit on the subway and free admission to numerous places as well as discounts on others. You can read it in detail here http://www.osaka-info.jp/osp/en/ it also says where you can buy it. Though when I used it, I picked it up from the tourist office in Osaka station near the Central gates.
It wouldn't be worth it for you to get a JR pass since you aren't doing any long distance travel and the travel you will be doing can be covered by other passes or just pay the normal fare.
Kyoto has a lot of history and culture but not all temples/shrines/palaces are built alike. I'd spend some time going through Japan-guide's Kyoto section and picking out a few that interest you by looks, history or uniqueness and try to find a nice mix.
I think you might like the walk from Kiyomizu dera through the Higashiyama distric towards Yasaka shrine. It will give you a feel for different type of Japan than is in Osaka. If you have time or inclination, the walk from Nanzenji along the philospher's path and to Ginkakuji is also really nice. Though I personally am not that fond of Ginkakuji itself, I do like the walk.
Another must see is the golden pavillion of Kinkakuji though it's off on the other side of Kyoto. You might combine it with Ryoanji which is in that area. And perhaps you'd either like to visit the Imperial Palace and the gardens or Nijo-jo which is a castle by name but is more like a palace in design.
I think if you don't want to feel too rushed and want to take your time to explore places spend 2 days in the Kyoto area, 1-2 days in Osaka depending on how much you want to see. Maybe combine a half day in Nara with either Kyoto or Osaka (It's easy to get to from either Osaka or Kyoto) and then a day being a bit more adventurous and going further afield.