Lilmod,
1. Who are these anonymous poskim? Rav Ovadia (Yechaveh Deah 5:4) says the opposite. Not only that he says that one cannot say “kim li” (I hold like others) as both Rambam (Hilchot Melachim 3:8 – where he says that a violator can even be executed by the government for minor offenses such as failing to make turn when instructed or violating house arrest) and the Mechaber (CM 369:6). Rav Shlomo Ishun even goes so far as to say that if taxes must be raised because of evasion the evaders are guilty of robbery (see העלמת מס by Rav Zalman Baruch Melamed on the Betel yeshiva website).
2. It think that they were pulling his leg. You and your brother are in good company though. I know an talmid chacham who has never been outside Israel. He actually believed an American who told him that there a red light is only a recommendation and not a law. I, in fact, know a rav whose sons and son-in-law have been in the army and he says the opposite. For that matter, I have a friend who did his miluim making a base kosher for Pesach (this is called מבצע פסח).
You are correct though that there is much room for improvement. The way to effect that is from within. Thus Rav Tau rebuked MK Smotrich for calling for guys to stop enlisting. The same goes for other areas of society. In fact, Rav Yaakov Ariel explicitly stated that it is a mitzva for those knowledgeable in both Halacha and secular law to become judges.
3. Many boys stop being frum in many places. Some stop in yeshiva because they are not suited for full-time learning but do not have any other alternatives in Chareidi society.