In 1779, land was given for a church and for “...a place of burial for said Congregation and Inhabitants.” The site contains graves of early Caldwell area families, including veterans of the American Revolution, the War of 1812, and the Civil War. Reverend Stephen Grover, the church’s first pastor and the man for whom President Grover Cleveland was named, is buried here. The church and cemetery are listed in the State Register of Historic Places.