Seminaries charge non-refundable application fees for the same reason that any enterprise charges anything for anything: because people pay it. If the fees were higher, the seminaries would get fewer applicants. If the fees were lower, the seminaries would get more applicants. Higher fees, therefore, give the seminaries fewer applications to wade through, and probably screen out the applicants who can’t pay the full freight.
There is a fair amount of halacha on pricing. Has anyone tried to apply it to the pricing of yeshivas and seminaries and analyzed whether the halacha is being followed?