i dnt think he means Most kollel ppl are wasting their time.
lets look at the difference between working and learning
work – ur held accountable for ur work performance, and u have the responsibility of providing for ur family which in effect keeps you working hard and putting all your energy into it.
Learning – ur on your own accord, and if ur learning performance is not what it shud be, no1 (except hashem) is going to hold u accountable. and since other people provide for you, theres no responsibility so it becomes a very easy choice for people,..
if ur learning just ask yourself, “am i putting is as much effort into my learning as the people who r proving for me r putting into their work”?