Hi Sam2.
You are correct in that notion.
Although idea of “proper attire” is not truly unique from person to person. Nor is the feeling one has wearing a muscle shirt and cut offs versus a tuxedo so different from person to person.
There is a reason bochurim are not permitted to go to yeshiva with their shirt hanging out, jeans, motorcycle boots, leather jacket, and possibly certain style sunglasses.
It does make a difference and have an effect.
Yeshiva boys are not like regular boys. They are on the level, to say the least, of royalty.
If they knock around in “street” clothes, I say it will have an effect on them. You say it won’t. Understood.