Sam2: I’m not sure what nekudah you are trying to bring out from that siman. All it is saying is that we are not makpid on other people’s clothing, not that there is no reason not to wear a certain color/type of clothing.
I don’t remember the mekokr for wearing white, it might be from the zohar since I remmeber him mentioning something about the Arizal. But it is probably based on the posuk in koheles 9:8.
In any case as I said before, whatever the mekoros are for wearing white, today we can’t do that since it causes gayvah in people that are not mushlam in their midos.