I would think that one of the issues here might be that different people have different ideas of what is considered “too sick to be around other kids”.
If the parents of the sick kid think he’s too sick to go to the babysitter, then I would think that they should be the ones who don’t get the babysitter. But if they thought it made sense to send him and the other parents didn’t, then it’s more complicated and I think you need to ask a sheilah. Maybe you should do that in any case.
I agree with DY that it’s best to work these things out beforehand, but that doesn’t help you now. Also, it’s hard to work it out beforehand since different people have different ideas of what is considered “too sick to be around other kids”.