We also know that there are many times we follow our leaders as if they were just about infallible. For example, when there was a sanhedrin, their decisions would be koveah halacha, even if completely wrong in an objective sense.
Ha gufa. Since they were kovea halacha right and wrong were irrelevant. Which is absolutely inapplicable to our time, when our leaders do not have that power.