Rejecting accepted offer and visa issues

I am currently living overseas, have never lived in Japan, and received an offer from a Japanese company - which I accepted 10 days ago. I have already sent them the information so they can start the process to request the visa. The job will start as soon as the visa is accepted, which would be around February-March.

Today, I received an offer from a different company, which pays significantly better (around 2 million more), but it is a rolling 1 year contract instead of a permanent contract like the first one. Start would be in March.

If I decided to accept the offer from the second company and told the first company I am not joining, could there be any issues with the visa process?

you can cancel the former one and accept the second one. the COE process will be restart again from the beginning. for your case, it is only 10 days difference, not much. no one can know which (the 1st one or the second one) is faster. it depends on companies.
if I say an advise, choose a famous company.

please make sure that you cancel the former one. otherwise, both of the applications will be rejected by immigration office.

Just for reference:

https://www.japan-guide.com/forum/quereadisplay.html?0+171784

You need to be sure that the first employer-to-be has cancelled their application first before the second one makes an application.