I did not mention 3 cases. You did. I mentioned 3 possible verdicts (and then cited the words of the gemara that say the same).
The 3 verdicts are possible as follows.
Chayiv misa. Someone killed with eidim and hasra.
Completly pattur. (This you didn’t mention) I guess, the killing was pure onnes.
One who is chayiv galus. This is one who kills “bishogeg” which is not the same as “ones”.
Again, from this braisa i do not see “if beis din can’t convict they hand him over to family”. If you do, which part of the braisa teaches it.