I think that Matan’s answer is correct in that they were making up for missing Sukkos that year and according to Bais Shammai the lighting of Chanukah neiros is somehow tied to the parei hachag.
However since we pasken like Bais Hillel the implication is that we don’t hold that the reason is parei hachag, bringing back the Bais Yosef’s question.
We can answer very simply, however, along Matan’s line of reasoning. Chanukah means a dedication because fundamentally the Bais Hamikdosh (and Klal Yisrael) was rededicated to Hashem. The Chanukas HaBais for the Mishkan was 8 days as was the Chanukas HaBais for the 1st Bais Hamikdosh (I don’t know about the 2nd). They therefore made Channukah, as a holiday of dedication, 8 days.
The mitzvah of neiros was to light one candle and l’mehadrin, to add to the parsumei nissa and kedusha, to add a candle each night until you have eight.