mentch, rav avigdor miller writes explicitly the opposite; that the amei haaretz were not even enough to constitute a class, and that if you wanted to insult someone in gemara times, you’d call them “reika” – empty one, one who does not have torah knowledge.
It wasn’t just vilna, it was in any major jewish city. the little ‘derfels’ had more amei haaretz, but this wasnt the majority.