Wear all your Shabbos attire until after Havdoloh. Some opinions hold even until after Melave Malka, but at any rate, surely until after Havdoloh.
This business of only dressing up for Shul on Shabbos morning, but not for Friday nite nor the remainder of Shabbos after services have concluded on Shabbos morning, unfortunately is very misguided & secular to say the least.
Shabbos garb is not to impress your fellow Shul goers, but to honor Shabbos & thus to honor haShem, and indeed, even if you are on desert island for Shabbos, the Halocho is very clear, that you must still wear Shabbos attire.