If the issue is with the time of day the calls are being received, and not opposition to receiving the calls altogether, it is a legitimate discussion. But other than not making the calls at an unreasonably late hour, how is the fundraiser supposed to know when you are eating supper?
Raising tzedaka is a very Jewish midda.