There are still a few apartments in Japan without bath/shower, so a bath facility is a necessity for many residents.
If you Google (in Japanese) bath facilities, you will find a note on Covid-prevention policies.
Here are a couple of illustrated examples I happened to find:https://www.ousama2603.com/about/tabid308.htmlhttps://onsen.nifty.com/onsen-matome/200716997916/
As you can assume, when entering the facilities, you will be checked your temperature, will wash your hands with alcohol and wear a mask. While bathing, you can take your mask off, but you should refrain from talking. So, the idea is generally the same as going to restaurants.
But a couple of days ago, I was watching TV when a popular comedian was showing his new resort home. He said that he had it built so that he could build a private sauna on his premises. He said that he would go to public saunas in the city, but other customers would chat without masks on, and he never felt comfortable about it. I've heard another man complaining on TV about people chatting inside a sauna, so I would assume it's not uncommon.
In fact, if it's regular bathing, the bathing room would be spacious with high roofs and windows slightly opened, and you'd also be free to walk out to the outdoor bath. But a sauna is a very small closed room, so I understand the concern. Have there been any reports of infection clusters originating at onsens, especially under conditions that did NOT involve tight contact in an indoor setting?
The thing is that most infection sources being reported today are either "unknown" or "at home". But I'm guessing that patients just don't give out sources any more, because they want to "protect" the businesses of whatever facility they went to.
But the other day, TV was showing an honest restaurant owner that he unintentionally created a cluster. You can see on TV that he always had all the necessary precautions perfectly laid out in his spacious restaurant. But, according to the owner, it so happened that a Covid-positive person was sitting right below the air-conditioner. The air from the conditioner was hitting that person before spreading the air all around the restaurant. I feel that information like these are necessary in trying to improve precaution policies.
On a related note, I think it's interesting that, under the pandemic, there seems to be more TV dramas on the theme of "enjoying solo".
For example, your question about baths reminded me how Megumi had put an end to partying with girlfriends she's not even interested in, and start adventuring to do things on her own, like bathing. She would then find herself striking quiet conversation with fellow solo people she would encounter. https://www.tv-tokyo.co.jp/solokatsu/story/
The 2015 manga "Yurucamp" (Laid-Back Camp), about a group of high-school girls individually enjoying camps often solo, was adapted into the second season of the anime, a spin-off anime, two seasons of live-action TV dramas all amid the pandemic, and a movie version is coming up for 2022. https://yurucamp.jp/news/information/6469
Of course, the protagonist of the long-running "Kodoku no gourmet" now wears a mask when he orders and takes it off as he enjoys his variety of solo lunches while imagining a great story on his own. https://www.tv-tokyo.co.jp/kodokunogurume9/