Shulchan Aruch Orach Chaim 75:1 talks about how tznious is dependent to your surroundings
Rashi in Kesubos 72a includes calf as such a case, Kaf Hachayim 75:2 includes calf and arms, and I believe Rav Moshe also includes calf and arms as depending on your community but I need to double check. ( They all say body/low neckline is always ervah).
Bottom line it is not completely clearly Assur by chazal, there are many more sources who hold arms and knees are not always ervah.