My personal policy (when I am in the mood of abiding my own rules) is to ALWAYS give, even if its a little, and even if its not because I think the beggar needs.
We ,especially as Yidden, must have rachmonius on ppl. When we ask Hashem to answer our prayers and look down , notice us, and give us a bit, we hope He is not going to look the other way and say “You dont really look truly in need to me”. Thats the midda of Hashem, to give. We ought to emulate Him.