There are three separate issues here, and I think that is causing some confusion, especially since there is some overlap and contradiction between the issues.
1) Punishment of the guilty.
2) Protecting the guilty from unjust additional punishment.
3) Protection of the innocent.
Punishment of the guilty: Let him serve his time, pay restitution, and so on.