I grew up with a guy who's eyes naturally look off to the side even when he was looking (from his perspective) right at you - a bit unnerving! Even knowing there's a uncontentious reason, you still got the same disrespected feeling that these shows are perpetuating on purpose.