It is a mitsvah to wear fancier clothes on special occasions, and velvet is fancier than regular cloth. It not “frummer”, its “fancier”, but sometimes there is a good halachic reason to dress up.
If you wear a plain cloth kippah on weekdays, than you can switch to velvet on Shabbos. If you want to look like you are wearing Shabbos clothes all week (e.g. to show you want to be considered prosperous and self-important, e.g. a lawyer whose respectability is his stock in trade), you might prefer velvet rather than plain cloth on weekdays.