Brim down means either you’re very yeshivish, faking it, or just don’t care about pretending to be cool and are wearing your hat like you’re supposed to.
That being said, I have a question for the oylam, especially pba who brought up shidduchim: What about someone who wears colored shirts and no hat even on shabbos, but wears his tzitzis out? How do you judge his avodas Hashem and shidduch prospects?