Its a tricky situation. You do want your seat but you don’t want to embarrass anyone either. As someone mentioned, the polite thing to do is to help the person find another seat. I, however, had an interesting situation many years ago. I was part of a somewhat new minyan which started Shachris on Shabbos at 9AM. I was generally In shul somewhere between 9 and 915. One Shabbos morning, I came in at my usual time and a nice newly married young man, who had just moved into the neighborhood, was sitting in my seat. I said nothing and sat somewhere else. After Shmone esrei, I walked over to him and mentioned to him that it was my mokom kevua. He offered to move, but I said it was fine, just stay for today.
The following Shabbos, I walked in, and lo and behold, this same young man was sitting in my seat again. So, what would you guys do in that case? I was extremely miffed, and I told him very firmly to never take my seat again. I felt it was a tremendous chutzpah of him to do it again.
Menu