“On the other hand, klal yisroel has adopted this story – when we start dismissing such traditions out of hand, we are on a slippery slope”
The reverse is true
when every made up nonsense gets elevated to Torah misinai, the line of whats real and whats not becomes blurred THAT is a slippery slope problem