at least in the eyes of those who don’t keep the Chumra.”
Seriously??? With all due respect, unless I misunderstood you your own statement seems to prove my point, if you yourself think the negativity is all in the mind of the meikeil. It isn’t. Those who hold by a chumrah typically tend to see themselves as more frum than those who don’t. It can cause problems with shidduchim, Sholom Bayis (when one spouse holds by a particular chumrah and the other one does not want to). Then, when the CHUMRAH becomes the ikar haDin in the minds of those who hold it, the actual Din becomes tafeil to the chumrah.