Whichever way you take this story, the bottom line is that gedolim started out like you or me, which is to say that we have the potential.
On that note, I went to sports games (Mets games) with my Rebbe when I was already in beis medrash. He was a big fan and enjoyed the occasional break (once or twice a season). Not everyone has to hold themselves to the level of a gadol, and we cannot disparage people for not taking on all of the ma’alos of gedolim.