Conditional love is not love. That is for your grocer and delivery boy: according to their output that’s how much regard you have for them. Shalom Bayis means a loving relatioship, not an un-screaming brush-by. If someone is inhibiting that, it should be dealt with.
There is some circular reasoning in the OP. If one is ruining the Shalom Bayis how can you ask if there is supposed to be Shalom Bayis?