There is mefurash S”A or Rama in hilchos tzadaka that says that we don’t ask a pauper to sell his house and move to a cheaper neighborhood. I’m not in front of a shulchan aruch but I’m sure others can find it pretty easily and quote it properly.
I asked this question when I found out that Har Nof has a nice sized keren tzedaka for residents.