I think the point of the Chazon Ish, is that men who are commanded to learn Torah, only are exempt from caring about Tzeniuskite like women, men only leave torah if they have to work, when they go out to work, they do not stay home, and cook, make the beds, sow, knead, clean the house, and look after the babies (mentioned in Kesubos-the woman’s jobs in the Talmudic times).
Over time social-economic changes have occurred in which women also have to work, and go to the shops, or go to school ect, so therefore in these generations ‘Tzenius’ is emphasized for women more then men, as women who are commanded to be completely hidden, now emerge out of unavoidable circumstances, therefore, they have to ensure there clothing covers the areas that need covering, they should be the ones to refrain from talking unnecessary talk to men, whilst the men who are commanded to learn all day or work should not need to worry about this specifically like women, as they are commanded to engage in 1 thing only.