OOMIS1105
In a way your right and in a way your wrong,
Yes you can notjudge someone wearing a blue shirt in a locale where bnei torah as well wear blue shirts.
However if you live in a locale that Bnei Torah do not wear blue shirts then one who so chooses is making a conscious decision not to identify himself with those who wear white shirts. When the one who sees him on the street then assumes he is not a Ben Torah he is not “judging” him rather he is acknowledging the decision the person himself has made.
In fact such things do have implications L’halacha.
There is a difference in Hilchos Yichud as to whether a man is considered a “Parutz” or not. Ashkenazim hold that every person is considered a “Kusher”. However Rav Shmuel Wosner (One of the undisputed Gedolei Poskim) holds that if someone owns a television then the person is rendered a “parutz”.
A prominent Posek in America told me in private conversation that as a whole he does not agree with Rav Wosners ruling in the location that he lives, However in Bnei Brak where Rav Wosner lives he does see merit in the ruling since it is a place where no person who is really a “kusher” has a television so by one simply bringing it into his house he is automatically showing that he does not have the level of a “Kusher”