If you read to the end of this from the ministry of japan. you will see you have to transfer it.If the holder of a double-entry or multiple-entry visa obtains a new passport while the visa is still valid, he or she can have the visa transferred from the old passport to the new one at an embassy or consulate. (The Ministry of Foreign Affairs in Japan does not handle this business.) Please note, however, that in this case the double-entry or multiple-entry visa in the old passport must still be valid.
In the case of foreigners who have already completed entry procedures and have temporarily departed from Japan with a re-entry permit from the immigration authorities, there is no need to transfer the visa to the new passport for re-entry into Japan, because the original status of residence in Japan is considered to be still valid.
Note: In the above case, foreigners residing in Japan, when they obtain a new passport, must have all valid stamps (e.g. landing permission, stamp of permission to extend their period of stay, stamp of multiple-entry permission, etc. obtained from an immigration authority under the jurisdiction of the Ministry of Justice) transferred to the new passport. For more information about transfer procedures, and/or inquire at the nearest immigration authority.