Wolf: Horiyos 13a if I recall correctly.
And Shein, the logic of a man before a woman is actually different than the logic of a Talmid Chacham over an Am Ha’aretz. In fact, a Talmid Chacham before an Am Ha’aretz actually trumps a man before a woman. A female TC (or, more likely, the wife of a TC) comes before a male Am Ha’aretz. See that Mishnah in Horiyos.