veHaKohanim veHo’om Hoyu Omdim bo’Azoro, so we do know what the people were doing:- They were standing in the Temple Courtyard.
They also felicitated the Kohen Godol when he emerged from the Avodo, unscathed.
Despite their fasting & the Bais haMikdosh not being air-conditioned, but the marble floors probably kept they Bais haMiokdosh fairly cool, indeed as the Mishno in Yumo alludes to with the Kohen Godol being fatigued & cold at nite, when young Kohanim flicked their fingers and read from book of Doniel etc.