The first Temple was destroyed (partially) because of idol worship, which is just like kefira. The second was destroyed because of sinas chinam. Considering that the second golus is much longer and more bloody, we see that sinas chinam is much worse.
The Gemara in Yevamos (62b) says that the talmidim of R’ Akiva died because they were not noheig honor one to another. It does not say that they felt disrespect, rather that they did not act in ways of respect. Having respect, while not showing it, is not enough.