IDP + DP from HK but French Passport ok?

I have read all the feeds about IDP but couldn't find an exact answer to this question. If I do have a HK Driving licence and a HK IDP (1949 Geneva convention) BUT a French Passport is it fine to rent and drive a car in Japan ?
Despite the fact I am French and holding a French passport, I got my driving licence in HK as a local resident. I just want to make sure it will be ok if the passport has a different origin than the driving licence + IDP (1949 Geneva convention).
Thank you for your great help as I have been searching hours for a clear answer on this matter...

I've never shown my passport when renting a car. The last couple of times I've only shown my IDP (Tsushima & Morioka). It probably depends on the rental agency and length of the rental. I usually only rent for a day - pick up at 8 am and return at 6 pm and pay with a credit card.