They’re afraid that whatever brought you to that place before will bring you to that place again (let alone the health risks you exposed yourself to that you’re now asking your future wife to expose herself to (or so they think, presumably).
There are those who will hold it against you, and those that won’t.
Any time a person declares a holier lifestyle, there are those who will not accept them because of their history. It’s part of life. MO know this when they date yeshivish. Baalei teshuva know this when they date FFB’s. Maybe your situation is to a more extreme degree, but it’s the same idea. You have to accept and deal with the given, and work on yourself to believe that H’ knew the path you’d take and prepared a shidduch for you.
Frankly, you might want to open your mind and heart to someone like yourself, regardless. Just like you wouldn’t want someone to hold it against you, why would you hold it against them?