CS, I do the same re:kippah as you do (except I was not elected or worked for civil service, you seem to have an interesting career), but somehow I do not have such visceral reaction to those who do not, even in our, supposedly more tolerant, times. Esav soneh Yaakov is still the halocha and some people feel it more than others. In some cases, it might be a menchlich thing to do. Say, you are a lawyer representing a client and the jury may be negatively affected by your yarmolka. I would think that your responsibility to the client would be to not wear it. Or, you are a doctor in an area, where people think Jews have horns. You may find that patients trust you more if you don’t have to show each of them that there are no horns under your headcover.
You did not answer, though, how is it tolerable to take Avira’s smicha away for the sin of teaching maths.