The easiest way is to get a marriage license from your local bureaucrat (it depends which county you live in), and have someone who can claim to be a clergy sign off on the mariage certificate. That way the government (including the IRS, Social Security, the health insurance, etc.) knows you are married. You can also file a notarized marriage contract, but no one ever does so. The government issued marriage certificate is what government agencies like to see when needing proof of marriage. There is a small fine for not following the normal procedure of getting a license followed by a ceremony – but lack of license doesn’t affect validity of the marriage from the governments’ perspective.
“Bedievad” – any religious ceremony counts, so you could have a huppah and kiddushin and you would be legally married under New York law (i.e. you need a civil divorce before marrying someone else), however it would require a lot of expense in legal fees to get a marriage certificate. While New York banned common law marriage, if one did have a proper “Kiddushin be-biah” it probably would be a valid New York marriage, but neither the rabbanim or the American courts would be happy with it.