He was referring to all men. You are an idealist. You think that: a) dating men should try not to have physical thoughts; b) because they shouldn’ t many don’t; and c) we should focus on those that act “correctly.”
The first assumption is partly correct, but the second is entirely wrong making the third one irrelevent.