If you take off Hol ha-Moed does it mean you won’t be able to go on a vacation in the summer/winter or won’t be able to go the mountains? If so, probably there is a real question whether you are allowed to work on Hol ha-Moed?
Can you make up the work without penalty or difficulty? If so, then you probably should enjoy the holiday at home.
Would you be out money? Is your job one that can only be done at a certain time and can’t be made up? In your field, are vacations fixed by custom or law (e.g. teaching in a public school). Then you probably have a heter.