“Talmidei Chachamim are exempt from building walls, etc., for shemira because the Torah itself is a shemira….”
Two points.
First, tell that to the families of the 1200 kedoshim who were slaughtered on Simchas torah.
Second, it you truly believe that All of the 80,000+ draft- dodging yungerleit are “Talmidei Chachamim”, you obviously are in need of some serious intervention.