Born enslaved in Virginia, Peyton Roberts became a freedman in Caldwell County, Texas, at the close of the Civil War. He soon moved west to this area, and the settlement that developed around his farm became known as Peyton Colony. In 1874, under the direction of the Rev. Jack Burch, residents of the community established Mt. Horeb Baptist Church. Early worship services were held in a log building also used as a school. Since the Reconstruction era, Mt. Horeb has served as a focal point for the community and for a widespread rural area.