Popa, the Gemara comes to that conclusion after asking why the Churban happened, if we know that they were such great Tzaddikim. If the Gemara is talking about the Saducees, they wouldn’t be called Tzaddikim, only, Tzedukim. Those who were fighting were far from Tzaddikim; they burned the food storages; they didn’t heed the Chachomim. Obviously, the Gemara is talking about what part the Frum people had in it.
It is usually understood that the story of Kamtza and bar Kamtza (hey, a great screen name for your next brother) is a demonstration of the Sinas Chinam that is referred to. I guess you can read into it that they were parts of different factions, but that isn’t the implied meaning.
The way I learned that Sugya was that since the Rabbonim weren’t Moche, they must have felt intimidated, themselves. That was the problem, that even the Chachomim had a connection to politics.