Shimon, looking for the exact source led me to an OU article, which lists a few sources for only drinking wine: Hitorerut Teshuva O.C. 3:491, Nitei Gavriel p.83, Rambam Megilla 2:15
The Rambam also notes that the drinking is part of the seudah, not independent of it.
כֵּיצַד חוֹבַת סְעֻדָּה זוֹ. שֶׁיֹּאכַל בָּשָׂר וִיתַקֵּן סְעֻדָּה נָאָה כְּפִי אֲשֶׁר תִּמְצָא יָדוֹ. וְשׁוֹתֶה יַיִן עַד שֶׁיִּשְׁתַּכֵּר וְיֵרָדֵם בְּשִׁכְרוּתוֹ.
Additionally, the Orchos Chaim says, “To be thoroughly drunk is completely forbidden. There is no greater sin than this, for it leads to adultery, bloodshed, and many other sins besides. Rather, you should drink a little more than you are accustomed to.”