“The OP is repeating what HaGaon HaRav Avigdor Miller zt’l said about this…
Well, not exactly……the quote below provides a bit more context…Fortunately, the Rav explicitly contemplated the likelihood that some would disagree with his political perspective and on that point, he was 100% correct.
“..There’s no question that the conservatives of today [circa 1976] are extremely more liberal than the liberals of thirty years ago. We have already advanced so far beyond the borders of liberalism, that today we can afford to retreat many miles behind these boundries and still remain in the forefront of liberalism. And therefore, today there’s no such thing as too much conservatism. You have to vote for conservative candidates on every level – on national, state and city levels……That’s my opinion. By the way, nobody here has to agree with me – on anything. Only, I don’t have to agree with you either…”
Source: TAPE # 133 (July 1976)