What to tell people? How about, “let’s garner zechus for Klal Yisroel and refrain from talking during davening!” “Though we don’t have the Bais HaMikdosh, the shul is a mikdosh me’at, and it’s our job to treat it respectfully.” “It’s hard to fully grasp it, but a shul is a special place, so let’s refrain from speaking.” Etc.
Chochma bagoyim ta’amin: Dale Carnegie in How to Win Friends and Influence People wrote that one of the ways to correct people is to begin by talking about your own faults. So how about, “Not talking in shul is a nisayon for me, too – maybe we can work on it together?” ETc. Take the idea and “run” with it.
Rabbi Yisroel Reisman shlit”a, in one of his recent Motza’ay Shabbos shiurim, spoke about the kovod one must accord a shul/bais medrosh . . .