Many people had dirt floors until a few generations ago. You’d definitely would have wanted to wear shoes then.
Rav Ovadia Yosef says that if the minhag hamokom is to greet respectable people w/o shoes, then you can daven w/o shoes. Based on a Gemara towards the end of Berachot (between 61 and 63). Clearly socks-only is not prohibited from halacha.