Throughout the entire period of the mishkan and the first Bais Hamikdash, idol worship was rampant. When Ezra returned to Israel after the first galos, he found widespread intermarriage and ignorance. The second Bais Hamikdash saw the influence of
Hellenism and the Tzidukkim. It’s nice to think of those generations as exalted but wishing it so doesn’t make it so.