Morah Rach- I find that very odd. If it’s so bad that you have to take off your Yarmulka you shouldn’t be doing it.
And if it was necessary due to emergency, then leave the Yarmulka on.
There is never a time in Yiddishkeit that you pretend to be a Goy.