Didn’t you say that your employer is frum? Even if the workplace is non-Jewish, or completely secular, you should be able to tell your boss that at the end of the day, when you were seeing the last patients, the other workers were leaving and you are not comfortable being left alone with a stranger. It is the employer’s responsibility to provide a safe working environment. He would be completely liable if you were harmed in any way.
Menu