“The main problem is their Kefira and that they don’t keep the Torah!”
Our Mesorah is to be good citizens even of countries based upon real Avodah Zara (idolatry) such as in the first Exile of Bavel. This Mesorah has been a key to survival of the Torah community.