This is a really tough situation. It seems like the only options are to hurt her feelings and keep your kashrus, or to compromise your kashrus. You could say that your husband/kids/wife/etc. want to spend some quieter Shabboses at home. That way you will get out of going there, but it doesn’t really solve the problem either. The bottom line is that she is going to be hurt one way or another – either by you telling her that you have problems with her kashrus or by avoiding them. Do you have to tell your Rav who it is? Will the Rav definitely figure it out? Do they have a lot of other guests? You may not be the first person who feels this way.
I have family that stopped covering their hair and I noticed some problems in their kashrus. To keep shalom (and for them not to accuse us of being the “meshugena frummies”), we are very careful with what we eat there. They often have a lot of people over, so we can get away with eating challah and cold cuts and staying away from “problem” foods.
I hope you can figure this out. Hatzlacha!