moish- (you’re putting yourself and the CR on an emotional rollercoaster here.)
JayMatt is right, your yetzer hara is doing a great job; and I can’t blame you for listening to him- he’s got years of experience on his side.
Beginnings are tough moish… change is even tougher. It takes lots of courage to switch direction- which is what you’re contemplating now. I can’t tell you “jump into it” if you don’t feel ready. But it does sound like you need this change- desperately. You can talk yourself into it or out of it.
It’s perfectly natural to feel this way. But as you talk it out and get used to the idea, the fear should subside a little. Maybe you should talk it over first with a Rabbi/mentor. I highly doubt they’d put extra pressure on you- especially seeing that you’re very skeptical to begin with. They won’t jump on you. A true mentor will only offer guidance and advice and you will decide for yourself.
At a certain point I’m sure you’ll come to the realization that you do want someone older than yourself involved in such a big decision- especially if it includes learning out of town. But take it slowly- there’s no deadline. Don’t panic. But keep in mind you really can’t go anywhere with this unless you speak openly about it. You can work your way in- mentor… Rabbi… cousin… father… maybe your mother. You will become more comfortable- this idea is really not strange at all.