There is a problem, but denying husbands the right to daven in a shul would result in no minyonim in many communities.
I was at a camp visiting day yesterday, and far less than half of the women were dressed according to a moderate standard of halacha.