Popa, I would feel comfortable in someone’s kitchen if I felt like they were trustworthy, and everything they did was halachically legitimate, even if I disagreed with some things.
So I would probably eat in your house. I would just make sure to ask all the relevant questions. As long as people have healthy attitudes towards others’ standards, there doesn’t have to be awkwardness.
The problem is, some people are not trustworthy, and/or their standards are not halachically legitimate, even if they consider themselves Orthodox.
So to the OP: if you feel this “rabbi” is not trustworthy, and/or his psakim are not halachically legitimate, then you absolutely did the right thing, and should definitely stick to your guns on this.