Very good point. It bothers me, too, to see the free for all attitude that prevails these days in this matter.
Having said that, let’s remember that we have a rule that Eilu Va’eilu Divrei Elokim Chayim. In practical Halacha it is impossible to follow both. Therefore there are rules of who to follow. In Hashkafa, the different aspects of all Shitos can be recognized. There are times when Chazal knocked out a certain Shita. In that case, as you mentioned, it is Assur to hold that way, since Chazal tell up it is actually wrong. But on an issue such as how old Avraham Avinu was who he recognized Hashem, we don’t Pasken like any particular Medrash. We understand that there are different aspects to recognizing Hashem, and Eilu Va’eilu.
Chovos Halevavos are not really just Aggaditta. It is a practical matter, whether you should believe in a certain matter or not.