I know that this question has been answered several times, but I wanted to provide a couple of real life examples I've encountered:
While I was in Japan, a group of six college students (myself included) had angered my (native) Japanese teacher by being a half hour late to our meeting point at the base of the Fushimi Inari Shrine in Kyoto and missing our first apology to the other students, because she was taking care of the train ticktets. In our defense, we went down the wrong path and cell phone reception doesn't work on half that mountain/hill area, but she was so incensed she said we should have called anyway! She yelled at me in English the entire way to the train station and onto the train as the other members of our misshap were cowering behind me, unofficially making me the representative of the group I suppose. I took such great pains to look at her in the eye while she was doing this while also spouting out appologies inbetween her pauses for breath. It is custom in America to look at your superior in the eye as a way of accepting your punishment and your responsibility, so I was trying really hard hear to show the proper respect so that she would calm down when she seemed to be just becoming angrier (she was so angry she was shaking). It was only later that night that I realized my looking her in the eye only incensed her further (she ranted for a good forty minutes). She's lived in the US for thirty years, but she was raised by her grandparents that took care of a temple in Hyogo prefecture, and she's very strict when it comes to teaching Japanese traditions and customs. I think in her stress and anger she may have forgotten the cultural difference of eye contact, as did I in my petrification. It's amazing how scary little old women can be sometimes!
However, that is a special case of an angry elderly woman that fits the obaasan stereotype perfectly.
Other than that one instance, I think I was more unnerved by some of the Japanese language tutors looking me in the eye all the time than they were of me looking at them. I felt as if they were staring rather than just paying attention to what I was saying. An example of a shy-ish American girl meets curious Japanese English club members. Insert more comments about my looks and appearance and it was just full of uncomfortableness for me. My specific tutors actually appeared overly interested in everything that I said that I felt it appeared insincere in some cases. I did my best not to take it to heart, perhaps they had been instructed Americans prefer direct eye contact? But add constant eye contact to constant ''nn, mm, aahs, sous'', and so on and it all becomes excessive. So it just goes to say that people everywhere are different, a few students tended to like direct eye contact more than I did.
In conclusion, I really think eye contact is almost the same as the ''western'' countries that prefer eye contact except for when speaking to a superior in a formal or one-on-one sort of situation such as accepting instructions (not listening to them but the final acceptance if that makes any sense), or apologizing, such as between me and my nihongo sensei. (Don't make my mistake!) In those instances, it has been better to avert your eyes in my experience.