I dont understand the Oval office comparison.
IT works for children, but for grownups it falls flat.
1) If I went to the oval office I wouldnt wear a tallis and teffilin
2) IF I was there 3 times daily for as long as I can remember and for as long as I have left on this planet.
You can bet I wouldnt wear a jacket
3) In a formal setting (assuming it wasnt 3 times daily) I would wear a tie and suit not a jacket
If you believe wearing a jacket is appropriate when standing in front of the Ribbono shel olam, then wear it. But not because of what you would wear in the oval office