To argue the other side, and I don’t mean to discount emunas chachamim in non-halachic/reshus situations, but I think it needs to be combined with an expertise in that area of “non-halacha/reshus.” A talmid chacham should not be dispensing medical advice if he’s a non expert.
Everything is in Torah, but only the greatest of the great are actually able to understand everything from the Torah. 99.999999% of Klal Yisroel, including the great talmidei chachamim, will not do well on calculus exam simply by knowing Shas, Poskim, Tanach, and Kabbalah.
A talmid chacham may be a talmid chacham, but if he is not an expert in medicine, business, politics, or even psychology, than his advice in those areas may not be the best advice… and not listening to them wouldn’t necessarily be a lack of emunas chachamim.