This is in Even HaEzer and Hilchos Ishus. What gives you the right to suggest this S”A and Rambam is merely Minhagei Chasidus? If you do this here you could throw out the entire S”A and Rambam by dismissing it as only Minhagei Chasidus. If you’re going to make this novel claim about a section of S”A and Rambam the least you’ll have to do it bring a halachic source claiming that rather than state it on your own authority.
Matan: Should we throw out Rambam because he lived in a different society? Rambam is bringing halachas (it’s called Hilchos Ishus), not societal customs of his era.