Just my experience that when people say things like what you did, they’re not just looking for mutual personal respect, but are bothered by people differentiating based on hashkafos. Like is it really so important if he’s a Zionist, or Lubavitch, [just using common examples, nothing personal!] or any sect that people may disagree with – and while you should personally be able to get along with and love him, you may have a real problem with them, and that will def. come out a lot on a communal level. Like no, I don’t want to formally invite a certain group to a specific function, because I don’t recognize their way as legitimate – and there’s nothing wrong with that.
Not saying that’s what YOU meant.