Sam, What I meant by physical is physical factors that affect a person’s choice. This is something we all agree to and deal with on a daily basis. Nobody thought to say that worrying about Chinuch violates the free will doctrine.
Likewise, we have a concept of ????? ??? by eating ?????? ??????. We also find that a person’s ?????? ????? are from Hashem. This we see in Rashi about Pharaoh that Hashem held him back from Teshuvah. This is also the famous Vort of the Baal Shem Tov that the point of the Bas Kol ???? ???? ?????? is that the person’s Neshama hears it and from there he gets a ????? ?????.
In general, a holy atmosphere causes holy thoughts and an Aveira is from a Ruach Shtus. Still, no one is forcing you to choose but they are affecting the surroundings.
The Ramchal writes that since a Zivug is the same Neshama, even before they know each other the actions of one affect the other. (I saw this a long time ago so don’t ask me where it is.)