The Village of Nelsonville derives its name from Elisha Nelson, who was born on May, 26, 1777 and died in 1852. Elisha is buried in the northwest corner of Davenport Cemetery, which is located near the front entrance to the Haldane Central School.

For many years Elisha Nelson was a tenant under the Gouverneur family. His first home was on the south side of the present Main Street, and east of the road leading to Garrison. Later "Uncle Elisha," as he was destined to be known as, built another home on land situated on the North side of Main Street, which is bounded today on the east by Spring Street. After World War II a new house was erected on this site by Mr. and Mrs. Harry Mekeel and still stands today on the corner of Spring Street across from Champlain's Blacksmith shop.

After building the first house, Elisha bought three acres and built another house a short distance to the west, and owned by John Lott. Other houses to the west quickly followed and were built by Joshua Purdy, Miles Burnell, Elisha Baxter, and Louis Squires. When the last mentioned house was erected a speech was made by Elihu Baxter upon the porch of the present 270 Main St., in which he named the growing community "Nelsonville."

A curious myth of local history was once told by the late Judge Chester Nelson, who stated that the name of the Village was actually decided by the flip of a coin, as the Nelsons and Warrens were the most populous families in the community at the time.