Did anyone check the sefer hachinuch?
The first explanation he offers is along the lines of what Sam2 said. The second, is that the kohen’s appearance might be a distraction.
It’s important to remember that any explanation is not complete, and can only offer a bit of insight.