Since the woman in question is Frum, a practical approach could be to tell her that you think there may be a Yichud issue according to Rav Moshe, and ask her to check with her own Posek to see if this is OK or not. That way, you’re not the “bad guy” telling her no – you’re raising an issue, and empowering her by having her ask the she’eila of her own Rav (assuming it’s someone reliable, of course). If her Rav says no, the issue goes away with no hurt feelings toward you – though if he says yes, you then need to speak with your wife to see if she’s OK with it, based on the Psak.
Good luck,
an Israeli Yid