In a case like this you may first need to have the property surveyed to properly establish that you own the land, and only then proceed to court to receive an 'ejectment order' to have the encroaching
improvement removed.
If you simply remove it yourself or find that the fence is really not on your property, you can be held liable for damages to the fence owner.
The idea to leave your name and contact information on the fence is a good one and a good way to attempt to resolve the issue without the courts and attorneys.