In a nutshell, if they’re not frum, they’re not family.
Anyone I know who have non frum family were told by their roshei yeshiva and rebbeim to keep in touch, to whatever extent they are able to keep their children away from negative influences. In some cases, based on the sensitivity of the family, they are able to visit, and in some are able only to host. But to dismiss outright as a rule is unacceptable.