If the guy is quickly fixing his Talis, turning his page or picking up a fallen Esrog, that is a reasonable delay. What else is he expected to do — continue walking while leaving his Esrog on the floor, his Machzor on the wrong page or his Talis falling half off??
Menu