a maggid shiur of mine once told me that sefardim have a mesora they come from shevet yehuda.
there is a little known vort from the chiddushei harim (or sfas emes – i don’t recall), who says a similar idea. it’s brought in siach sarfei kodesh, and there he says ashkenazim come from shevet binyamin and sefardim come from shevet yehuda.
lmaiseh there has been so much intermarrying between sefardim and ashkenazim throughout the doros, that it’s impossible nowadays to say for a fact that a particular person comes from a particular shevet, just because they’re sefardi or ashkenazi.