Kinda not.
How does a cross-dresser wearing it indicate that it’s a woman’s hat? Just the opposite. The fact that the women’s rights movement adopted it as its symbol also shows that it was considered masculine.
Also, look around some more and you’ll see that similar hats with different names were around before.