“even Rav Moshe says it should only be eaten in a shas hadchak and that a “baal nefesh” shouldn’t eat cholov stam.”
Rav Moshe would never matir TREIF in a shas hadchak or allow someone who is not a “baal nefesh” to eat TREIF. So certainly Rav Moshe held chalav stam was kosher in all instances.
However, everything else being equal, i.e., there are two containers of milk in front of you; one is cholov yisroel and one is cholav stam, Rav Moshe suggested that you go lifnei m’shuras hadin and drink the cholav yisroel. But if the chalav stam is freash and the cholav yisroel is not, or the cholav yisroel is finished, he permitted to drink the cholav stam…l’chatchila.