It’s one of those asinine expressions, where the questionee doesn’t know the answer but thinks s/he sounds smarter if s/he doesn’t admit it. This expression is even worse than “let’s agree to disagree” or the like, because it attempts to portray the question as being faulty.
Let everyone be reminded that in the Talmud stories were constantly questioned.