One thing I am, admittedly intuitively, confident of. There is a lot lot lot more unproductive freeloading loafers who sit behind a desk in an office offloading their responsibilities to colleagues (and anyone who worked in a large organization/company as I have sees them all the time — and amazingly getting away with it), then their are people in Kollel not trying to learn their best. I am, again intuitively AND from observation, confident there are very very few of the latter especially compared to the former.
So lets direct the bulk of our criticism to the freeloaders in the office, rather than the constant criticism of our holiest Torah learners, while you hear extremely little criticism – comparatively – in our circles of the former.