It depends on the woman. I had a very bad experience with a male gyno, in fact I could probably sue he was so unprofessional, and i now try to avoid male ob/gynos as much as i can.
You are right that many woman are okay having male obs but many aren’t. However, even for the ones who are okay with it, they would never be comfortable if they knew him from outside the office.