Anyone is free to disagree with me but P.E. in Japan is at least extremely more important than it was in the elementary schools I attended in L.A., and that's how all people I've personally spoken to see it.
In the two public elementary schools I attended in L.A., P.E. was not much different than an ordinary recess. You were encouraged to participate in whatever the teacher told you to, which was almost always softball or volleyball, but you were hardly forced to. No one really taught us the rules, so for a person like me who is bad at sports, be it physically or academically, P.E. was just boring. We did P.E. in our ordinary clothes, so doing sports while wearing skirts didn't give us much opportunity either.
On the other hand, in schools in Japan, P.E. is much more organized. Everyone in any public school have to change to P.E. uniforms, so you have no excuses. Everyone is taught a variety of gymnastic and field sports, and in summer you'd learn swimming. In the recent years, traditional sports typically judo has joined the curriculum. Until the so-called "yutori-kyoiku" began, you were pretty much forced to achieve a certain level, which for many people nurtured the ability to overcome difficulties.
Sure you can get bad grades after the 2 to 3 classes per week of P.E. and still go to college (where often you'd still have another year of P.E.!), but you can't flunk it if you want to graduate. Moreover, since P.E. is done in your youth, those who are bad at P.E. tend to be seen as an annoying person, because you annoy your fellow participants of team sports. There are also tests, and you also need to participate in the Ball Sports event as well as the Sports Festival.
So it may depend on your definition of "important", but just as friends are "important" or wearing the right clothes are "important" for a school boy/girl, P.E. is very important in Japan. And the best thing that can happen to people like me when graduating high school is that you don't have P.E. classes any more.
The following link is just a backup.
http://www.mext.go.jp/a_menu/shotou/new-cs/youryou/syo/tai.htmhttps://www.bunri.co.jp/infosrv/shinkatei_s.html