If you really wanted to leave, you wouldn’t be asking us. It seems to me that you want to try to make this work. It may or may not work, in spite of your best efforts. Decide what is most important to you; what your priorities are. I agree that you should make a list of pros and cons about this place, and see what adds up. If you feel your davening is compromised, but your learning is good, that is one issue to weigh.
The fact that this is a college-oriented Yeshiva is NOT necessarily a bad thing. Realistically, today’s yeshiva boys have to accept that the money trees are blooming less and less, and the newer generation of young parents, who were supported by working (outside the home) parents, do not have the financial wherewithal to finance the next generation of boys who want to sit and learn. You need to be educated. Everyone needs to be educated, even if only to be able to deal with people outside of our daled amos. We do not live isolated from the rest of the world. Now is as good a time as any to recognize that, or you are in for a rude awakening.
If your chavrusa is MIA, find another chavrusa ASAP. If the bochurim are interested in stuff other than the learning that they are there to be doing, then why not try to be mashpia on them in a pleasant way (not heavy handed and judgmentally). People do need an occasional break. If in the end you feel this does not work for you, then you have some hard decisions to make about finding a yeshica more suited to your needs.