Is it rude to leave Sumo before it ends

Hello,
I will be attending a sumo match during May in Tokyo, my group will have to leave before the event concludes is this considered rude?

No, it is not rude. But you will be missing the best wrestlers and most exciting bouts, which are held at the end of the day. Especially if you leave before 4pm, you would be missing most of the spectacle.

People wander in and out all day, it's not rude at all to leave. As Uji says though, after 4pm is the best time to be there. The wrestlers early in the day are still relative novices and often not all that impressive to see. Try to see at least a few of the makuuchi (upper level) bouts after 4pm if you possibly can.