Yosef is not one of the patriarchs, Avos. There are only 3- Avraham, Yitzchak and Yaakov.
It is actually interesting, because despite Yaakov’s preference of Yosef over his brothers, and giving him the firstborn’s right to a double portion (his tribe being split into 2 and getting 2 portions in the Land of Israel), aftwerwards, other shevatim become more prominent- for example, Levi as the leaders in Egypt after the passing of Yaakov’s 12 sons, culminating of course in Moshe and Aron, and kehuna. And then Yehuda, the tribe of royalty, and of course, Mashiach. They are represented in the ushpizin too- as Moshe, Aron and Dovid.