I am lucky enough that my work gives me off on chol hamoed. Not everyone is that lucky. You need to speak to your rov and see what what he says. It may not just boil down to getting fired. It may be better to work then and take off another day so that you and your wife can have quality time. Not everything is black and white.
