The danger on both meikel and mahmir side is to make sure you are not deluding yourself when making decisions. We see here lots of people, including myself, who will not change a position on an issue even when offered a strong argument against. So, then, how can you trust yourself?
According to a Rav who errs on the strict side himself: if you commit to strict halakha, you can usually look up a sefer and you will know what to do. If you commit to humros, you are in danger. Should you, for example, do another humra at the expense of your family? I did not have a chance to ask him about an obvious way people handle that – join a community and buy books where not following humros is considered gehenom and your questions go away.