Or perhaps the idea is that it is always possible for someone to convince you that their way is much higher than yours and that you are practically a Chotte for ignoring their holy Avoda. “You think it’s Avoda Zara? Ach! What do you understand? Come, I’ll explain everything.”
It sounds reminiscent of how the Christains through the ages would chastise us that we are doomed for not believing their belief.