Mamashtakkah. I suppose it is for this simple reason, yeshiva bachurim have also adopted not to wear Khaki pants.
There is little room for error if you have closet filled with white shirts and black pants.
I wonder if white shirts and blue pants would pose a problem for the non zionists.