The Granville Cemetery Association was organized in February 1881. Its incorporators included Civil War veteran Adin H. Potter, formerly of Granville, New York, who deeded the land. During the war, he was one of the few Union soldiers to escape from Andersonville, Georgia, Prison. The earliest marked grave dates from 1880. Many interments were infants and children, including Edith, daughter of Adin and Diana Potter, who died February 18, 1881. Francis H. Baker is the only known Civil War veteran buried here.