It is evident from the Third Perek of Pesachim and from the Chochmas Adam 92 that minhag hamakom has precedence over minhag avos, that is also why 1 or 2 days YT depends on where you currently live. While it is true that Minhag Yerushalaim is exaggerated, in this case there is a confluence of the Sefardim, Chassidim and the Litvaks who were talmidim of the Gra and there IS a near unanimous minhag not to wear tefilin Chol HaMoed. Therefore one who is a permanent resident of Yerushalaim MUST follow minhag hamakom and stop wearing tefillin.
Please bring a source for this teshuva from the “uncle of the Beis Yosef” because the way you quote it has numerous inaccuracies.