@RSo I know there’s a ton of criticism against him, and many Litvishe Rabbonim blamed him for the decline of much of Chabad into a bizarre messainic cult while he was alive and well, but I can’t say I’ve heard anyone say he wasn’t a Tzaddik.
One example: A very very Chareidi Rebbi I knew was giving his requisite “College is terrible” speech, and used as an example “Only three gedolim ever went to college, one started a apikorsus institution, ones followers became a weird Moshiach movement, and only one was OK”. (He was referring to Rav JB Soleveitchik, Rav MM Schneerson, and Rav Yitzchok Hutner). As a Yekke, I asked why he excluded Rav Breuer, and he said that he wasn’t really on the same level (not sure if he meant in terms of gadlus or influence).