My Rosh Yeshiva would say that the kedusha of shabbos keeps rising until it’s over, and the zemer of Yedid Nefesh is only able to be sung during seudas shelishis because we can only attain such a high level at that time.
B’kitzor this is one of the holiest times of shabbos and we should show it the utmost respect with our clothes.
While it’s true that we don’t give shalom and we say tzidkoscha, but it doesn’t say anything about removing begodim. Adarabah I would say the we don’t give shalom and we say tzidkoscha is because we want to show some form of aveilus without taking away from shabbos.