Eizeh shtuyot. I’ve already posted the Rambam who says that not working and only learning and relying on the public is a most honorable and noble thing to do.
Also, there is no such anti-semitic canard about jews. The canard is that we do work and do outdo everyone and get all the money. The canard you refer to is an anti-religious canard started by other jews, who in truth are very similar to anti semites in many ways.
And if everyone would do like them, then G-d would still figure out a way to feed us, as the Rambam says, that when a person becomes like shevet levi, they don’t need a chelek in the land to earn a living because Hashem is their chelek.
I have no problem with you not learning full time (I certainly don’t learn full time), as long as you are willing to keep the Torah, which includes supporting talmidei chachomim and not being a mevazeh talmid chochom.