The way I see it: when someone meets a married friend, they say, “How are you, how are the husband and kids?” Etc. So for an unmarried friend, they say “IY”H by you.” Wouldn’t you say the same if the roles were reversed?
But if they give pity faces, of course that would bother me. I don’t pity myself, so don’t pity me.