Great in theory, but for the vast majority, it doesn’t work. They need other pursuits, and if there’s no secular studies program, they’ll find some less wholesome activities to engage in.
DY: That is your own speculation. Since there are few schools that eschew secular studies, there is little evidence for you to compare students who did not attend secular studies to those that did, and determine if one group engages in less wholesome activities than the other group.
Furthermore, as I pointed out, in Eretz Yisroel, almost all Chareidi Yeshivos do not have secular studies at the high school level. And you will be hardpressed to claim that the yeshiva boys in EY are spiritually worse off than the boys in the US. I would in fact say the opposite is the case.
At first thought I would agree with your point that a shoddy secular studies program is much worse than a competent secular studies program. (OTOH, perhaps a shoddy secular studies program takes less after-school hours/time from the students, thereby allowing them more time to engage in Limudei Kodesh.) So you might, indeed, have a point that it would be wrong to bad-mouth a yeshivas secular studies program.