Learn the Maharsha that I referenced in my earlier post in this thread. Of course we know that people travel (including, by the way, Avraham Avinu, the paragon of Kove’a Makom L’tfilaso). The point is that by having a set place to daven, one expresses the fact that HKBH is All-Powerful, and above any sort of influence whatsoever. This is why the Gemara says “Elokei Avraham Be-ezro.” The G-D of Avraham (who was the first one to teach about the omnipotence of Hashem) will help a person who davens in a way which expresses this idea.
Traveling out of necessity is one thing; traveling so as to daven in what is considered a particularly auspicious place undermines the message of Avraham Avinu.
Again, I want to stress that this is not me talking – it is the Maharsha.
[It would seem that this applies only to mandatory tefillos, as we know that all Gedolei Hadoros were (and are) in the habit of davening al kivrei Avos and at the Kosel.]