Presbyterian congregations formed early in the county’s settlement, with one established in 1803 on Col. Casey’s farm and another in 1827 in Columbia. The church was built in 1857 and has held continuous services ever since. The county and city congregations merged in 1912, and Columbia-Union Presbyterian Church became the official name in 1925. The original sanctuary’s slave balcony was removed in 1885. Civil War connections include bullet molds found in the attic, a steeple lookout used to watch for rebel raiders, and doors used as stretchers for the wounded in a skirmish with John Hunt Morgan’s troops in 1863. The limestone steps are original.