If I’m understanding you correctly, the idea that one rishon (r. Moshe tako, a rishon who’s never mentioned in halacha as far as i know. His statements were ignored by the rishonim like rav yosef albo, the ritva and tons others who take it for granted that it’s a braisa) And a karaite both claim something makes us believe it, because certainly if two distinct scholars agree on something, there must be some truth to it. Enough for you to use the term forger as a given.
Let’s unpack that. You’re assuming that a karaite scholar has some measure of credibility or weight. Would you say the same if a rishon and a Christian/muslim/buddhist scholar agreed? You’re also assuming that this one rishon can overweigh every other riahon and achron who by omission and by quoting perek shira completely disregard this shita.
Braysos are torah shebaal peh. Claiming that one is a forgery is probably under the category of machish magideah as per the rambams list of defining apikorsus. And no, I don’t throw around the term easily.
People who harp on dayos yechidim when it makes them comfortable in biblical or Talmudic criticism, yet shun other shitos as “daas yachid” when it’s something they care about are the personification of intellectual dishonesty. One of my rebbeim used to say that the maskilim, who enjoyed researching the source of texts far more than actually learning them, are “modeh bemiktzas begalui and kofer hakol b’seser”.