Benignuman –
Firstly, how do you define Histaklus without Hirhur? More specifically – what exactly is Hirhur? If I admire a woman’s beauty but have no sexual thoughts – is that hirhur?
Secondly, maybe the answer to the Stirah in the Rambam is the point I originally made.
I.e. that Vlo Sosuru applies to Arayos D’Oraysoh and the Issur Histaklus that applies to ALL women, even non-ervah, is D’Rabbanan.