Cantor –
The key word is refined. If you (and your community) feel what you’re wearing is up to par, kol ha’kavod. But if you’re wearing it with an in-your-face attitude, thats another story.
My shul is frequented by many visitors, as we are walking distance to both BP hotels. And yes, the occasional guest wallks in without a jacket (and sometimes even in sandals w/o socks!). But if I were to do that, I’d get my head handed to me.
Bottom line: if you feel like you’re dress code is appropriate, its your call. But what if the next person (or perhaps your own child / child-in-law) wants to take id down a notch to where THEY feel comfortable? That’s why we have standards. And for the most part (where its a grey area) our gedolei hador set those standards. OU/YU may have different standards than Chabad or Satmar, but they have one common denominator: They try to look like their zeides and babbes, in both deed and presentation.