Gemora has a lot of conditions that describe Talmidei Chachamim, such as “merabim shalom b’olam”.
If you dismiss these issues just because other people are worse or “they started first”, seems like you are treating “yeshiva” as just “my team”, which kind of misses the point. Rav Salanter was holding the opposite: if a professor in Paris is spouting apikorosus, it means we in the yeshiva are doing something wrong.