Real-brisker,
The difference is that (presumably) going to college increases your wage potential. You are more likely to contribute to society both in terms of your skills/job availability and tax money.
A person who sits and learns in Kollel does not increase his earning potential or skill set in any way that is tangibly helpful to the American public.