Joseph: All of Klal Yisrael is Raui to share one Korban Pesach. Veshachatu Oso Kol K’hal Adas Yisrael. We are not Karaites who take the simple reading of Pesukim instead of Torah Sheba’al Peh.
(Although, just to remove a possible Stirah, it probably is Davka by Pesach Mitzrayim that you needed a Seh LaBayis because it was Assur to leave your house on Leil Pesach to eat at someone else. In fact, this answer is too good and simple that is has to be a Gemara or Chazal somewhere. Anyone help?)