I would not focus so much on mitzvot or lack thereof or his observance currently being “less than mehadrin”, because as I said, I think it would be the same if it were a secular family, then he’d seek excuses for avoiding sunday brunch or the baseball game or whatever else. This is one additional reason I believe it is a mistake for you and your husband not to step in and take responsibility. Please, keep in mind he is very vulnerable.
If you feel you are lost, I believe you should go to your rabbi, the one you trust, and talk to him and then do what he says. Nobody, not even a qualified person (and I am not, nor do I know if other CR commenters are) is able to give sensible advice over the internet. Please, don’t rely upon us. I know we are anonymous to each others and the rabbi knows you, and we all, in your position, would rather not. But, we all sometimes have to say embarrassing things to the rabbi (or the doctor, or our spouse, or our parents, or worse it was found out by others), and yours does not even compare to some worse ones we’ve had to share and that I can, unfortunately, think of.