Yserb, actually, many say that the gra and the rambam did their secular learning in the bathroom. The gaon said of himself that he found only a few minutes of time during an entire year that he didn’t learn enough.
He also wrote that to the extent that one is missing in secular knowledge, that will impact how much he doesn’t understand torah. Yet he does not argue with the…. Again, shulchan aruch iron clad rule, not to make it a set study.
When you learn torah 1 percent of the time and level of the gaon, then you can worry about not knowing enough math and science