Dash: I don’t think that’s a problem. The Achronim point out that it’s only something that could break (or break you) if it falls and you would be worried about it and it would hurt your concentration.
DY: I’m currently trying to think out the way along the lines of what the Achronim bring down that if you see a Sefer on the ground and it bothers you you can stop your Shmoneh Esrei to pick it up. There should be some way to make this situation similar to that, but you’re right, it needs some creativity.