Personally, I can make an argument that Rabbanim should wear black hats, as it is part of their uniform, just like policemen wear caps, etc. But when I see little 8th grade boys at recess in black pants and white shirts playing ball, I wonder what this is really all about.
At any rate, the American chareidi levush is rather mild, but the chassidic levush I have to admit, causes me some embarrassment. If I were talking to a non-Jew, and a chassidic person in full levush came up and started to ask me something in a broken English, I probably would feel some embarrassment. Maybe I am a bad person.
It is this idea that some groups seem to believe that there is bdavka a mitzvah to give off an appearance of being primitive, and not being able to speak the language of the country, not learning anything secular after Bar-Mitzvah, not even basic algebra, dressing in such a different manner, that I suppose I would not be honest if I didn’t say it doesn’t cause me some level of discomfort. It is not that I don’t love every yid, but is this really what the RBSH wants us to appear like in the eyes of the world? Does this fulfill Ki hi chochmaschem uvinaschem l’eini hoamim? Doesn’t gemara say they would not appoint somebody to the Sanhedrin unless he knew 70 languages? Also, I think that the gemara said that Rebbe Yehudah Hanassi hitiru lo lsaper kumi, they allowed him to get a real goyishe haircut, since he had to interact with govt officials.
Anyway, there is certainly a maylah of separating ourselves from bad influences, but somehow it has to be balanced with the need for us to maintain a dignified image.