Staying married will cause more hatred. Get devorced and love the Ger from a distance, like the rest of us who aren’t married to this Ger.
The Gemara in Kedushin, second Perek, says that Ve’ahavta Arei’acha Kamocha is a reason not to marry someone you wouldn’t like. I’m pretty sure Ve’ahavtem Es Hager works the same way.