AAQ, shu”a does NOT differentiate between casual moshav leitzim and consistent, nor do the nosei keilim; it’s an issur like any other. Gambling is a different issue, because with aidus we need trustworthy people. If someone casually gambles it doesn’t take away from his credibility, either because he is not always owing people money and will take bribes or the pshat you quoted as not being part of productive society.
Moshav leitzim is the very act of assembling for something other than legitimate tzrochim or avodas Hashem. It’s a chilul Hashem, because these people could be serving Hashem and instead they’re wasting their time openly.
It also applies privately to secular novels (shocker) as stated explicitly in shu”a, that they are assur not only on shabbos but during the week as well, if they have no spiritual merit.