Marrying J-spouse, but remaining in USA

Hi all,
My fiance ( a Japanese native) is living in Japan right now and I live in the USA. We both visit each other twice a year (he comes twice to the USA and I go twice to Japan). We want to get married, but I cannot join him permanently in Japan until a year or two later. We want to be married but living apart until I have things ready here.
My question is that if we proceed to marry, can I continue to visit Japan on a tourist visa? I never even stay 30 days. Likewise, can he still visit me in the USA on the visa waiver program?
Also, should children come into the picture, do I have to have special permission to take our child out of Japan to visit my family in the USA?

My question is that if we proceed to marry, can I continue to visit Japan on a tourist visa? I never even stay 30 days. Likewise, can he still visit me in the USA on the visa waiver program?

Yes, your visa status (or actually lack of any visa status other than visitor) doesn't change until you apply for one. Most likely a spouse visa is in your future. Incidentally, I've been looking into the procedure for both lately and the Japanese spouse visa is a cakewalk to receive compared to the American version.

Also, should children come into the picture, do I have to have special permission to take our child out of Japan to visit my family in the USA?

Nope, your children can be duel citizens until they turn 22 so they would be more free to travel between the two countries than either of their parents. You on the otherhand will just need to make sure to get a reentry permit before you leave, otherwise you will invalidate your visa.