The department of buildings wherever you live should have precise property lines on record. Your mortgage company probably had a survey done too. If not, you can have pay for a survey and have your property boundaries clearly marked.
Have you spoken to your neighbor about their construction? Perhaps they are not aware that the contractor they hired has placed things on your property. Who knows, they might apologize and have it moved. They might ask for permission (which I would grant on the condition that they signed a legal documeny stating that any damage resulting from their use of your property is their responsibility to fix). If they say, we know and don’t care, then remove the wall, or move it back onto their property.