If I know a man is Jewish I try to avoid the situation as well.
I work with a conservative man. I am able to avoid shaking his hand because he sort of understands. My boss is half Jewish (the right half) and I can avoid that too.
But random people I don’t know? How would I know they are Jewish? Especially if their name is not traditionally, ethnically Jewish?