“I’m really curious wear the source is that says swimming with one’s wife IS assur.”
Are you curious abotu where it says its assur to eat pork with yur wife? Or say loshon hara with yur wife? Or wear shatnez with yur wife? It doesnt say it becuz its assur all the time. So is mixed swimming. It doesnt matter if yur wife is with you or not.