technically it could be anywhere, so art least we can throw out the kotel…that’s waaay too close.
See the teshuvah from R’ Moshe referred to above, where he says that in theory, there are parts of HH”B which are muttar, but we don’t know where they are (partially based on a machlokes Rishonim), and that would include the Kosel (because perhaps it’s an interior wall), except that we have a mesorah that the Kosel is muttar.