“An interesting point from this would be that if someone who is makpid on Chalav Yisrael was invited for a dairy meal by someone who drinks chalav hacompanies, they should have no problem with the milk/cheese being used.”
According to Rav Moshe, when one b’davka keeps C”Y 3 times, s/he is making a neder. That crosses the line from chumra to Torah law.