People all the time rebuke those who go to far with Kulos, but rarely if ever do they rebuke those who make too many chumras. The damage can be equal.. There is a slippery slope when you permit things that should be prohibitted as people might go down that road
However the same applies in the opposite way. If you prohibit the permitted people will find out it was just some chumra and that you lied and begin to question other things you have said, Maybe they were “Chumras” too