Have you thought that maybe I worked late and didn’t have time to go home and get my hat. If the choice is not davening or davening without a hat, I think the answer is rather daven without.
Also, most yarmulkes have a lining and can possibly be considered a double covering.