US Secretary of State and General Colin Powell was a Shabbos Goy in the Bronx. And he was proud of it. He pridefully retold the story of him being a Shabbos Goy for the rest of his life.
That Goy was proud to be a Shabbos Goy while our OP is ashamed to have a Shabbos Goy.