Stop providing communal cover for it, and we’ll stop assuming that you condone it.
Would you make part of your religious community someone who drives on shabbos? No.
Someone who publicly steals? No.
Someone who publicly eats pork? No.
Someone who publicly molests children? No.
So why do you make part of your community someone who “eats dairy out”? (Assuming he doesn’t understand the issues like yitayningwut)
So why do you make part of your community someone who doesn’t cover their hair, or wears low cut necklines?
And it isn’t just about being more “live and let live”, because you certainly wouldn’t accept someone who did other issurim I mentioned. It is apparently about thinking these issurim are no big deal.
And that’s what you’re being called out on.