If in fact you don’t want them parking on the other side, then that is why they are mad at you. It isn’t your land.
I look at it this way: The next door neighbor can allow the across the street neighbor to park, but it would greatly inconveinience the OP, so meiheichi teisa to do a favor at their expense, especially since the OP does favors for them.