Allow me to elaborate on shein’s point: Imagine four people, A, B, C and D. (Imagine A represents our Yishmael.)
A marries B and they have children 1 & 2, C marries D and they have children 3 & 4.
1 marries 3. 2 marries 4. All of their children are descendants of each of A, B, C and D.
So although other people existed in A’s generation, all of the people two generations later are A’s descendants.
If there were 10,000 people at the time of Yishmael, there is only a 1 in 5,000 chance that someone from the immediately following generation would be a descendant of Yishmael’s (each member of this generation would have 2 parents from the preceding generation, thus multiplying the odds pretty much by 2). But with each succeeding generation, the odds of being from one specific member of that original generation increase dramatically. And there have been at least a hundred generations since Yishmael lived.
Another way to show this: You have 2 parents, 4 grandparents, 8 great grandparents, 16 great-greats…etc. So by the time you get back to Yishmael’s generation, each of us likely descends from pretty much every single person who lived on the planet at the time and had children.