Columbia Primitive Baptist Church was formally constituted on the first Sunday in October, 1833, after serving for more than a year as an arm of Bethany Church. Late in October 1833, Moses Dees was Columbia’s first delegate to the annual meeting of the mother church, Union on the Alapaha River. Earlier, on May 3, 1833, while Columbia was still an arm of Bethany Church, Thomas Newbern deeded the church property to Samuel T. Henderson, Moses Dees, and Jarvis T. Frier as trustees for Columbia Church. The first edifice was a small log house across the road from the present church, later replaced by a larger hewn-log building with a puncheon floor. A more commodious edifice was built in 1872 and has since been improved. The Rev. Matthew Albritton was one of the earliest pastors and served until 1847. Columbia Church was originally a member of the Ochlocknee Association and remains so.