It depends on whether they’re sane or not. Not all serial killers are shotim. Having a yetzer hara for something which you could, if you chose, control is not the same as a paranoid schizophrenic who hears voices telling them to kill. In fact, most paranoid schizophrenics are harmless. I’ve known a few, and some of them are actually likable. Only a tiny minority are violent, and those are not usually not mentally coordinated enough to get to be a serial killer.
The real question is what to do with a serial killer, and that’s a no-brainer. You lock the person up permanently, and make sure they don’t get out unless, after many decades, they’re certified harmless. In terms of the Beis Din shel Ma’aleh, that’s not for us to make cheshbonos. There is only One Judge, who knows the real motivations and will act accordingly.