“what is considered beged ish/isha can change based on societal norms. For example, he told me that pants are no longer considered beged ish.”
Fedora hats were originally beged ishah (and a political statement in favor of licentiousness) but today they are widely accepted in the frum world.