What I dont understand why you cant force him to either go to School or get a Job?
It is still your house and you have the right to tell him that he cant just hang out all day
You are being soft IMO, Dont force him to go to Yeshiva Gedolah, but force him to do SOMETHING productive. You can even give him a choice, College, Yeshiva or a Job that way he wont feel trapped