Moshe Rose, it is very nice that you bring this mekora from this sefer. If you believe in this sefer and the mechaber, then you should definitely follow everything he says. That being said, it doesn’t mean that your belief in the mechaber must transfer to everyone. If anyone has a sheilah about this or wants to know if he/she can go swimming with their own spouse they should ask their own Rav or their own Rosh Yeshiva.
This here is just a blog and not a place to pasken halacha, so each of us knows what we know and unfortunately we are sharing, but we are not rabbonim or dayanim and we are not allowed to pasken for one another what is proper halacha or not. So before you start having shalom bayis issues with your spouse, go to your own RAV and ask your own sheilos. And while your at it, ask them about speaking to your wife too.