At my shul, there is always the ability for me to move as not every seat is taken. That being said, I would probably consider the following:
1) There probably have been times when I disturbed others in the same way, and I was oblivious to the effect I was having on others. I regret any and all times that I have done so.
2) The people may be “once a year” visitors to the shul. If they are shushed by a regular, they will form an opinion of the relative friendliness of the shul, and this will provide them with a reason not to come on a regular Shabbos.
3) I might not recognize the person but he could be a generous contributor to the shul, and helps to keep the lights on.
4) Will my shushing be heard by more people than their talking? Really, this is an issue, since to get their attention, you have to be louder than they are.
While the Gabbai can’t be everywhere, I have seen people going to the Gabbai when a group wouldn’t knock it off.