Feif Un,
Valid point. To my mind, if it is a hefsek to answer amen, then the chiyuv to answer falls regardless of whether the bracha was made out loud or not.So again, I am not sure why saying it softly helps here. But the minhag was and still is widespread.