“Like Haman would not have noticed some random person not bowing “
Maybe. But it’s been widely reported (on YWN http://www.theyeshivaworld.com/news/headlines-breaking-stories/259251/rav-aharon-kotler-zatzal-truman-and-hirohito.html for example) that Rav Aharon Kotler was beaten to a pulp for not bowing to Hirohito. And remember that Mordechai would have been in a position of prominence as one of the Jewish leaders.