I don’t even know what you mean by taking six hours out of their day to celebrate. Where I live, you are either somewhere within an hour away, or you are traveling in from another state. If you are traveling from another state, you will either come in the night before, or that morning. Whatever time the bris is, is when you plan your ticket. Most people don’t fly in for a bris and fly out. Most people don’t drive three hours for a bris and drive out, so I have no reference point for a scenerio where you would need to adjust your bris time for an out-of-towner.
And if I was driving two or three hours for a bris, it would NEVER occur to me to ask the person to rearrange the whole minyan time for me because I want it to work for my travel schedule.