As far as beautiful scenery goes, there are better times of year to go. But, you can still see some good places.
Visiting the Yutoku Inari Shrine in Saga is an excellent idea though. It is one of the best places in Kyushu, yet nearly unknown to many.https://youtu.be/2UU_1Aip_l4
If you are driving, there are other good places to see. One other little-known gem is the Arita Porcelain Park, which is like suddenly walking into Germany.https://is.gd/U0NzDz
Plus there is Yoshinogari, which is an ancient archeological site of the Yayoi period (c. 300 BCE-300 CE).
It seems like you like hot springs a lot; if so Saga has some as well like Takeo Onsen, frequented mostly by the locals.
If you are going to Kitakyushu at night, then one very nice place if the weather is good is Sarakurayama, with a spectacular night view over the city.https://youtu.be/Hl1pCW9Ibbk
Please be aware however that with the JR Northern Kyushu Pass you cannot
use the bullet train between Hakata and Kokura.
For Aso, if you're driving and near the Uchinomaki onsen, it is not far to drive up to the Daikanbo Lookout, Nabegadaki Falls, and Kikuchi Gorge.
And regarding Fukuoka itself, one of the finest places is the Nanzoin Temple, just 20 minutes by train from Hakata Station. Within the city, there is also the Tochoji Temple, with Japan's largest wooden carved Great Buddha, plus some nice old gardens and villas like Rakusuien, Yusentei, and some scenery on Shikanoshima.
The castle ruins are not bad either, and the Fukuoka Tower gives a great view.http://www.thejapanfaq.com/fukpic.htmlhttp://bit.ly/1KAm8cU
Plus if you want something for the 2 year old, there is an Anpan Museum there too.http://museum.anpanman-acm.co.jp/en/