Interesting. Likely.
Really, the camp didn’t articulate their reason very well. The real reason is they want the counselors to be role models generally for the hashkafa they want to teach, and presumably you didn’t have those hashkafos. I don’t know why they said a stupid reason like seeing you in pants on an off day.