This happens in my family all the time. I and the Mrs adopted cholov yisroel, my parents and sisters did not.
So when invited for a milchig meal, the host goes thru the effort of not serving us (nor cooking with) cholov stam. They do NOT use different pots or cutlery (yes, we eat on disposable plates).
After all, its not like they cooked traif; its a chumrah and we chose to adopt it, so they accomodate it. Not much differnt than someone who is counting calories or limiting their sat fat intake. You don’t have to be obnoxious about your chumros. If you’re upfront about your needs, people will try to work with you.