The ‘dislike’ you mention is a vestige of Hisnagdus. Although Hisnagduss itself is mainly gone, the psychological barrier remained. As a result, most Litvishe Melumadim are, for the most part, completely ignorant of Chasidishe ideas. Therefore, anyone who was close to a non-Chasidishe Rebbe who heads toward Chasidus will get a funny response.
On the other hand, the term you use, “learning Chassidus”, is usually used by the ones who kept vestiges of anti-Litvishe sentiments. That might be what is feeding your attitude.