Some points:
The words in this Medrash are indeed about the guy Eisav. For some reason, we’re all aware that this has been taken as a generality.
We all know there is such a thing as חסידי אומות העולם, and there’s no reason to think that it is a handful in a century. And if there are חסידי there are likely some that are just below that as well.
On the other hand, western cultured people are formal and know how to behave, but this doesn’t inform you what they’ll say in private. Sometimes it can come out after a long time.
In essence, it isn’t far from what we find that it is called Har Sinai because משם ירדה שנאה לעולם. We find in מסכת ע”ז the opposite: that there is a secret admiration. We do see that the world is obsessed with us, one way or the other. And so, we aren’t looked at passively. Most people have opinions about the Jews. Either we are respected or repulsed. It is often both. (Just like a man hates his conscience…)
While Europe is understood to be a continuation of Rome, and by extension a part of Eisav, America might be different. Baruch Hashem, we never had anything close to what our grandparents went through. And in Europe today you still feel the hate.
In closing, the term doesn’t have to be a general rule for all time, or has been taken that way by many and that’s how we are aware the quote, and it isn’t a large leap from what we find elsewhere.