Nobody argues. There are no possible reasons, and that one is not true. The reason is that
(a) devarim sh’beleiv einum devarim. The buyer has a right to all that stuff, and is not supposed to know whether you have a preference towards not selling real chametz.
(b) the presumption is absurd, since nobody would rather be oiver on baal yera’e baal yimatzei than sell real chometz.