sushe: You are picking a narrow subset of places and times. Most towns in most times were not rich enough to have anywhere to put them.
Consider the story of chatzkele l’kavod shabbos. He grew up on the street, late at night he came to the synagogue and said teach me something. So he knows how to read and recite the sounds. He knows the beginning, but he doesn’t know the end.
When will we go again to yerushalayim, and when will we sing again? In the holy temple, l’kavod shabbos, oy l’kavod shabbos, l’kavod shabbos, oy l’kaaavod shaaabooos.