till you get to letters like ches /chof, tes / tof, vais / vov straight, it can drive teacher and student to a standstill.
The fact that we pronounce those letters differently is likely a product of exposure to other languages, not the actual pronunciation. I believe that the Teimanim distinguish between them.