As “myfriend” says, the problem is even seeing the mixed dancing or improperly-clad women.
Men looking at women dancing or at improperly-clad women should be avoided whenever possible, even in the street, as it is damaging to the Neshama.
A man who watches women dance violates the prohibition of Histaklus (observing immodesty), which is termed by the Mesilas Yesharim as Znus Ha-Aynayim.
It’s also a shame that Pascha Bchochma’s father had to come up with an excuse for not dancing. If an Torah-observant person chooses not to join in, he should not have to make an excuse to less-observant people, if he’s not stopping anyone else from dancing.