No, you should not feel guilty.
The fact that others have it worse makes absolutely no difference to you. If it is hard for you, then it is. Telling yourself that your feelings are illegitimate because others have it worse is not healthy, or what Hashem wants.