From what you just posted, the problem is far greater than the absence of relations. The man is not observant! How can you contemplate staying in a marraige like that? (I take it that you yourself are frum).
Sorry to say, but if his frumkeit is shaky, his fidelity most likely is as well.
Stop kidding yourself, thinking that you can’t do better, and that he is doing a fine job. Give him a 6 month window to get his act together and if he does not, move on.