As I'm Japanese, I don't know about the social worker thing, but I would imagine that people scream because entering a house with shoes on, for a lot of us, is basically the same as stepping into a place where you would sit flat on the floor with footwear that might have stepped on dog shit on the way. I'm not exaggerating.Also, outdoor shoes in Japanese is "dosoku" which literally means "soil feet." We have a saying, "dosoku de fuminijiru (dishonorablly step on with outdoor footwear)" which means to dishonor horribly. To step on something with shoes where you're not supposed to has a nuance of putting mud on someone's face. I don't know the origin of this saying, although I naturally share the feeling.
All this will be excused if the place was meant to accept shoes in the first place (as in western style hotel rooms or places like Eiji's home). In places like that however, I would literally frown at people who eat food that was accidentally dropped on that floor. I'd say, "Yuk! That floor might have been stepped on by shoes that may have touched dog shit!" (which you have to admit is true).
Of course, when Rome I do as the Romans do. Whenever I travel to certain exotic places, I switch my mind and accept that everything is basically the same as the toilet floor, and if I should get sick, that's because I was too weak.
Btw, stepping on tatami with shoes is out of the question no matter what style of living you choose, because it would just be too difficult to clean the dirt or maintain the woven surface if you step on it with hard soles. Would you accept people who step on your pillow or a woven straw chair with shoes on? I think not.
Also, I doubt that Shinto has nothing to do with this, because again, Thailand shares quite a big shoes-off culture. And my bible teachers said that the reason people in the bible kissed Jesus's foot is because feet were considered as the most dirty place of the body and to kiss that would be the best way to show respect.