Depends.
A famous, well respected Mashgiach in a major NY-area yeshiva went to college at his parents insistence. When he asked his Rebbe, one of the gedolem, which college he should attend, out of the two options, his Rebbe recommended a Catholic university. He asked him Rebbe, “But there is a huge cruifix on the wall of the lecture halls!” His Rebbe answered “Are you afraid he will jump off the wall and hurt you? In this environment, if you spend the rest of your time in yeshiva, you won’t feel pressured to fit in.”
There is nothing that is black and white in life.