Can I apply for CoE when old one is unused?

My previous employer got my CoE in March 2020. But due to travel restrictions, I could not go to Japan and the CoE is still with that employer. That document was supposed to be expire by June, but it has been extended by government until April 2021. My prev employer is unable to bring me to Japan even after October 2020's new rules and have postponed my joining indefinitely. Now I have got a new offer and the new employer is wants to apply for fresh CoE. But since my last CoE is still valid, do I need to withdraw the older one?

I believe so.
As far as I know, two CoE applications at the same time would be rejected, as the applicant obviously cannot fulfill the two job assignments at the same time (for example, for a work visa, in your case).

Because the first employer-to-be in effect withdrew their offer, they will need to inform the immigration authorities that they are no longer going to be the sponsor, and then you�fd need to get the new employer-to-be to start the CoE process.

But you might want to first explain the situation to the new employer-to-be; they �gmight�h have some expertise/experience hiring non-Japanese people and might know of resources like immigration lawyers.