> if we let the religious people spread their religion they’ll go after us.
I think you are both right. R Soloveichik asks why we have issur against avodah zorah in 10 comm, but not about atheism. He answers because there are no true atheists, they all end up believing in some type of AZ. Pretty much sure he was thinking of commies and nazis.