“The reason that mixed swimming is assur is because it is kiruv l’arayos.”
Do you know that thats the reason? Or maybe its assur because its among the things one should not do. Just to repeat the point, just because its with your wife doesnt mean that its permitted-there are plenty of things that are forbidden even with your wife.