Can we swim or dip in Fujinomiya pool?

I did not see anyone in the pool of the waterfall.
Is it because it is cold or is it because it is not allowed to enter the pool of the waterfall?

What about other waterfalls?
Is there a rule that stops visitors from entering the pool for dipping?

Are you referring to Shiraito Falls?

http://www.japan-guide.com/e/e6368.html

I'm not sure about that location specifically, but some places prohibit you from entering the water below waterfalls for various reasons (it dangerous, the pool is sacred, etc.) in which case there may be signs. If in doubt, go with the flow with what other Japanese visitors are doing.