Not saying amen to a bracha is quite serious. what gives someone the right to purposely ignore 19 brachos and not answer amen, even if they ignore the bracha by opening a sefer.
Oh, and as far as people spacing out. See the same M’B cited above for the reason one should not learn, even if will be careful to answer every amen.