The serugah people can not have it both ways. Some of them put it on as a symbolic statement of sorts, then demand that you recognize no inherent difference.
Someone once went to Rav Shteinman and showed him two Yarmulkes, one regular black velvet and the other seruga. He asked if there is any difference. Rav Shteinman replied that there is no difference. The fellow then complained, “My son’s school wants to expel him for wearing the seruga!” Rav Shteinman responded, “So if there’s no difference, why doesn’t he just wear the black one?”