tbh, its hard to say without knowing the contours of this driveway. if they share it in some way, then as long as he didn’t park their during his neighbor’s time there is no adversity (and, incidentally, no notice either – they often go hand in hand). this is all baba basra stuff: if you have some kind of permission (an umon or aris) such that the owner doesnt have reason for alarm, you can’t adversely possess.
also, I think ny has some weird color of title laws. watch out.