The congregation at Little Zion dates to c. 1832, when enslaved individuals worshipped in white-controlled churches. After emancipation, these former slaves founded independent congregations like this one. A group led by Doctor Entzminger purchased the land where the church now stands c. 1886. In earlier years services were held under a brush arbor, and later a frame sanctuary was built. In the past 130 years, Little Zion has had only five pastors. The longest pastorate was Rev. Abraham Chandler, who served for fifty years from 1887-1937. Under the leadership of Pastor Eddie W. Davis, the newest edifice was built in 1995 and added land was purchased. In 2016, a part of Winnsboro Hwy. was named for him in honor of all his accomplishments and service as pastor since 1983.