What Milhouse is saying is only true in cases where it might cause embarrassment. And, I could see people be strict even then.
If somebody invited you and then said, “hey, I know you’re stricter with kashrus in this regard, so if you don’t want to eat this it’s fine,” then you wouldn’t eat it. You’re talking about shalom bayis heterim, not some system of halachic relativism in which you automatically convert to the shittas of whomever’s house you happen to be eating.
As a proof to that, while it’s true we don’t treat those who hold by bad eruvin as posul from eidus, you still aren’t allowed to benefit directly from them carrying on Shabbos. Their shittah is not actually considered to carry water (no pun intended); we just interact with them in the way that will cause the least embarrassment sometimes meaning relying on kulos.