Wow, this is a tough situation.
My friend has this all the time at work. She is afraid to wear anything new or different, because of the slew of comments that would follow. I had to beg her to get a haircut!
You know what? You are your own person! Be strong and do your own thing man! Just smile when the commentary begins and show your self-confidence.