When I was much younger, my father used to step out for about 5 minutes (really no more) to have a lechayim with a few of his friends. I went with my father once and he joined a few men and they made 1 lechayim, had a piece of harrying on a cracker and went back in time for Av HaRachamim.
My father even found a Rishon (I do not remember who) who said that in his time it was the custom of the young people to “taste liquor” before Musaf. So the custom goes way back and if done in a respectful manner, it is not the end of the world.