Even the punishment of the kipa, I believe, was fairly limited.
For example, take the story of R. Shimon ben Shetach who saw someone chase a second person into an alley. When he finally reached the alleyway, he saw the murderer standing over his victim with the bloody knife still in his hand. However, he expressed his frustration that since (a) he didn’t actually witness the murder and (b) there were no other witnesses, he could not have the person face the justice he deserved. At that point, a snake came along and bit the murderer, killing him.
The upshot of the story, however, is that R. Shimon ben Shetach was frustrated because he was unable to do anything about this fellow whom he knew with 100% certainty to be a murderer. Had he been able to take him to a Bais Din and have him punished extra-judicially, he would not have felt so frustrated as to express his anger at the murderer.
Hence, it seems that Bais Din could not use extra-judicial killing whenever they wanted to. There must have been policies and rules in place limiting it’s use to particular circumstances, and that this case did not meet those circumstances.
The Wolf