I have actually heard this question asked to a Rosh Yeshiva of a large NY area yeshiva, with regard to young Kollel couples. He responded that couples having other couples over for meals IS something to be concerned about. He mentioned both issues — it leads to socializing with the opposite gender, and it can lead to shalom bayis issues due to comparisons. (This is especially true with younger couples who are still becoming comfortable in their roles. Additionally once there are older kids at the table the men/ women socializing issue is less)
Despite this, he felt there were certain advantages. (Not Hachnasos Orchim which, as has already been discussed, applies to someone in need of a place.) He felt that it is particularly important for women to have a “social life” (my words, not his), and often it is difficult during the week when holding down a job and caring for young children. Therefore his recommendation was that it is ok to do, but couples should not repeatedly have the same people over and over, which creates more intimacy and opportunities for problems. Rather they should rotate and invite different couples each time.
Of course each person should speak to their own rav or mentor for personal application.