MANY MANY MANY moons ago I spent over almost 2 years living and working on a religious kibbutz.
Though I was never a black hatter (though I do sometimes wear a black hat on Shabbas) I was, at first, shocked at the way the guys dressed on Shabbas. They wore a white shirt, no tie, and nicely pressed chino type pants (that could be black or navy or olive or beige, etc.) and maybe a cardigan syle sweater, or in the cooler winter weather they wore an army style bomber jacket.
At first I thought, hey, this is a little TOO casual for Shabbas, even for a kibbutz.
Then I came to realize that THOSE were the clothes that, for them, were their good, SPECIAL Shabbat clothes (as opposed to the navy blue work shirts and pants most guys wore during the week.
So, my point is, it is a matter of intention, of kavana, what makes you dressed appropriately, dressed Jewishly.
On kibbutz, the clothes I described, along with a nice bright, clean kippah seruga (as opposed to the ones that get dirty looking from working in them all week) is JUST AS Shabbas’dik as a gold striped kapota and streimel for a Yerushalmi, or an 800 dollar suit and a new Borsalino for someone in Flatbush.
Bottom line…
Al tistakail ba’kankan, ela b’ma she’yaish bo.