I don’t think the judgement is necessarily applied to normal ‘poor people’.
I can picture it in a situation where you want to decide if a person claiming to be poor is telling the truth. For example:
On every Wednesday (I don’t know why specifically that day) I am faced with 20+ collectors during shachris. I cannot afford to a) give all of them a decent donation, and b) cannot interrupt my davening so many times. I therefore feel the need to dismiss/ignore some of them if possible. Almost all of them are ‘regulars’ who come every week.
There is one in particular who wears gold rings, a fancy watch, and drives a very nice car.
Does that mean I am complaining that he is ‘not poor enough’? Rather I assume he is not as needy as the other people that I choose to give my money too.
It helps that he is rude, does not know how to answer Amen and does not even know not to ask people for money during Shemone Esrei. Maybe the next CR discussion can address whether they are “not Jewish enough”!