“You can’t throw out the window a bifeirush gemara, the Rambam, poskim, the Maharal, acharonim, the Chofetz Chaim, the Satmar Rov, Rav Elchonon Wasserman, Rav Aharon Kotler, the Brisker Rov, the Chazon Ish, et al.”
You have yet to sight a halachic source, a teshuva or a code, from “Poskim” or Acharonim, that say that the oaths are binding l’halacha. The Rambam in Iggeres Teiman is not saying that the oaths are binding l’halacha. There is no issur in trying to take back E”Y by force. The Rambam is rather saying that attempts to do so by Messianic pretenders are doomed to fail. Rambam is saying it is foolish, not assur. If they succeed on the other hand . . .
In learning Torah Sh’Bal Peh one has to be able to differentiate between aggada and halacha, between drush and pshat, between rhetoric and substance. Each has its place, but if you mistake one for the other you will be in error.