Many Litvaks learn chassidus, too. Some openly, some secretly. I know a “closet Breslover” in a kollel, for example. Others can be found in Uman.
The Litvishe rabbonim I know love to give over chassidishe torah in Shabbos morning drashos.
This isn’t anything new. Reb Shraga Feivel (a big rav before WWII, founder of Torah uMesorah) was yeshivish. Yet he studied the Tanya, Likutei Moharan, etc. And Rav Hirsch to boot. He learned everything and taught it to his students too.
Pick up an unfamiliar sefer now and then and see if it speaks to you. You may realize you had a void that needs filling after all.