The fact that most of them live in New York and New Jersey.
We are all human, and we unfortunately succumb to our desires frequently, whether we realize it or not. Included in those desires are the unhealthy interpersonal ones, such as jealousy, peer pressure, creating subgroups of people/places/practice that are "better" or "worse" than what you know, and the list goes on.
When we live in highly concentrated areas, and deal with multitudes of Jews day in and day out, we inevitably start creating subgroups of better or worse Jews, schools, shuls, etc. We create more and more "options" for things because we feel the need to keep splitting ideologies apart. The more options there are, the less success we'll have living a healthy, wholesome lifestyle.
The Torah is the source for how living a happy, meaningful life is discovered through creating limits and boundaries. When we only have 1 shul to daven in, 1 school to send our kids to, and a small, motley collection of Jews we can associate with, we will have more wholesome, meaningful lives.
The solution: The Frum Jewish population should be spread out across the world in communities of no more than 100-200 families, and most importantly, the people grouped together should be selected at random.
I welcome your thoughts.