Think of how telling someone you can’t be friends with them will affect their view of the frum community. As a yet to become Baal Teshuva, I strongly feel that the traditional Jews really need to work on righting their outward image of being unfriendly.
What would Rabbi Hillel or Baal Shem Tov say?
Instead of pushing them away, ask them to be part of your community by inviting to your Shul or for a Shabbas meal.