The little I know
Yes they were these huge “pilgrimages”
The Beirach Moishe of Satmar spoke about the huge crowds etc when he went to the yortzeit of the Zanzer Rov together with his grandfather R Shulem Eliezer of Ratzfert. There are also pictures open things of the yortzeit in Zanz.
In Kalev ,where the Beer Menachem had there was a big yeshiva , and there were many talmidim who survived, including the Tosher Rebbe ztl (who brings it in his sefer) it was well known it was like lag bomber and all the Rabbonim and Rebbes were there definitely from the whole of Hungary but from other places as well.
The Divrei Yoel Spoke about it as well.
In Ziditchov it was said that a hundred thousand people came to the yortzeit. So even if only a 10th actually were there it’s still ten thousand people not just immediate family
In Ihel and Dinov Riminov it’s well-documented in the Seforim of the Rebbes who went there and brought down in many seforim.
So just because you heard from a few people who never traveled out of their little town that it was so doesn’t change the historical fact the same way for some reason you are trying to make as if just a few people went to the Rema.