From what you’re describing, the not letting yourself in a nutshell = fear of committing.
As long as you didn’t know he was serious you were easygoing and enjoying yourself. Now that it apparently is up to you, you’re afraid, because if you fall for him emotionally, that means marriage. Marriage is a big deal, so your defense mechanism kicks in.
I’d say go for it, but I’m not wise or informed enough to say so.
Good luck either way!