Firstly, Rava argues.
Secondly, if this approach was reasonable (that there’s hidden rishus the whole time), why did Abaye need to say that Yochanan was Yanai, he could have said that he was always a hidden rasha.
Thirdly, I think you misunderstand the Tos. Yeshanim (see Ritv”a) but either way, why wouldn’t that be a reasonable explanation of the Chozeh’s shamash’ behavior?
Al Ta’amin does belong in the discussion.