How about not worrying about how long everyone else is davening SE and just worry about your own (however long it takes)?
If it’s a small minyan, davening extra long might delay chazaras hashatz and hold back others from getting to where they need to go (e.g. work in the morning).