Most likely the water temp will be around 22 deg CThat is not true. The water temperature aroud November 20 is usually 25-26 degrees around Okinawa main island. And another 1-2 degree warmer around the Yaeyama islands further south.
So, the water temperature is certainly warm enough for bathing.
Average air temperatures are as follows:
Main Island:
Daytime high: 24 degrees
Nighttime low: 19 degrees
Yaeyama Islands:
Daytime high: 25 degrees
Nighttime low: 20 degrees
So, on some days, when it is not sunny, it might become a little bit cool for enjoying the beach. On other days, it should still be enjoyable.