If one thrapist didn’t work, try another. I know of people who failed by a few therapists until they got the right one. Anyhow, ending it would be discussed with a professional, as well.
Even if your behavior is offsetting the situation, both of you will need consultation. It’s possible to love someone who has a tough disposition. You just have to learn how to understand, and relate to, him. Not every therapist will focus on this. It is a more natural response to address the behavior that is being perceived as aggressive (or whatever the issue is) directly. Being that it won’t change overnight, a better approach is to learn how to deal with it, and how to appreciate someone who is working on it.
I really hope I don’t come across as condescending. Sorry if I do.