Most English teaching jobs require you to be currently living in Japan, but some also require that you already have a work visa.
If you see this requirement in the advertisement, that means that THAT company is not willing to take the trouble to sponsor anyone for a visa (for cost or for administrative consideration, or from past experiences, whatever it may be). So you will have to find an employer who is willing to sponsor you for a visa. Precisely because some people come, find a job with an employer who sponsors them for a visa, and leave soon after to find a new employer, some are hesitant to sponsor new people for visas.
Even in the circumstances you described with JET, if your initial visa sponsored by JET was limited to a year, for example, then after the year is up, (before the visa expires) you will have to find a new employer who is willing to sponsor you at least for a "renewal" of the visa.
Please check the websites suggested by Sira :)