You think they stay up at night figuring out ways to harm chareidim? They have many other things to think about, some of which are important, like how to earn a living, how to afford an apartment, or how to be safe in a crazy region, how their children are doing or how they are going to pay to fix the air conditioner that just broke; some of which are not important but entertaining or amusing, like who won kochav nolad, how to get to the beach tomorrow, or what that new restaurant is like. They care about responsibilities and privileges that affect Chareidim and they feel should be equitable, and they care about their individual rights being respected and infringed, but beyond that they frankly think about you way less than you imagine. And the vast majority of secular Jews are simply not ideological.
+1