Why must he go? Because he did a bad personal thing? So what. If his politics are sound, keep him. If not, that’s a reason to fire him.
His personal life? Of zero interest to me.
And yes, I know he lied. All politicans lie.
For the record, I have no clue where his policies stand. But his personal life is of no concern at all. If anyone should call for his outster, it should be his wife. Not the public.