“Yes, I have learned that, but that does not include R”H , where the Torah says that one should be with his family.”
Where does the Torah say that? R”H is not one of the 3 regalim, and halachically there is a huge difference. According to the Ari I believe one should not “be” with their spouse on R”H as it is a Yom HaDin.
Chassidim have always left their families to travel to Rebbe’s on RH. If one has the zchus to have a Rebbe he understands that his entire inspiration and hadracha for Yidishkeit comes from that source and the minhag was always to spend R”H with that Rebbe. It is silly to start asking why don’t they go to XY & Z. Those people are their direct influence right now the same way a Satmar or Skver chassid goes to their direct influence for R”H today as well. Yeshivisheh people even go to their Yeshivos many times and I know several who felt as if they need to leave their families to travel abroad to their Yeshiva where they learnt so many years ago.