Those who consider the whole house as Daled Amos only quote the Shita of Rebbi Shimon ben Elozor who said that a room is like Daled Amos — by a Meis.
The Daled Amos comes from a Zohar Hakadosh that the Beis Yosef quotes. Otherwise the basic Halacha would require that you wash your hands before Davening.