There used to be a pop singer in the 80’s (not the one who died recently) who used to wear a tuxedo out of respect for his audience, even though he wasn’t singing anything respectful. Don’t Hashem, His Torah, and Shabbos deserve a similar respect, to be feted with bigdei Shabbos instead of golfwear? It is also making a provocative statement when the minhag hamakom is not to dress that way for Shabbos. If I walk down 125th Street in Harlem wearing a white bedsheet and a white hood, saying “But kabbalah says I should dress in white!” doesn’t take away from the fact that I am making a provocative statement, wheras if a toddler walks around in a white bedsheet he’s just playing “ghost”. One shouldn’t judge others by his clothing, but someone who goes out of his way to violate social norms has to be viewed differently.
There is a midrash where Hashem says that when little children read “Ahava” (“Love”) as “Eiva” (“hate”), He loves it, even though the children completely messed up the passuk and said you should hate Hashem. If an adult purposely misread the word, Hashem wouldn’t find it so cute. The difference is that the adult should have known better . That means that the exact same action can be performed by 2 different people and can be legitimately viewed through different prisms. A baal teshuva, ger, or out-of-towner who dresses that way is clearly not making the same statement, and has to be judged more favorably.