Do the right thing, you’re heart is definitely in the right place, but what happens when a person believes that someone has gone beyond the boundaries of the Torah? Do we then say, well, he has lots and lots of followers so we should stay out of the discussion?
If that were true how would anyone have known which side to take when Sh”Tz came on the scene (again, I’m not comparing, just illustrating a point)?
Despite the greatness of Rav Avigdor Miller z”l, I think DaasYochid put it very nicely.