I would put up a sign outside my door “Baby Sleeping, do not ring the bell or knock on the door” and/or in Hebrew or Yiddish, the same.
You also might want to invest in an intercom with a camera (good idea for security anyway), so you know who is there. I go to my window and ask who it is, because otherwise I would have to go down to open the door. And if my husband is not home, I NEVER open the door anymore. At night when they sometimes show up, even if he is home, we do not open the door to ANYONE we don’t know. We tell them to return in daylight hours, but they never do.