Settlers began occupying the Baxter area in 1850, and schooling was at first informal and community-based despite repeated efforts by the Texas Legislature to establish a public school system. The Alligator Public School began classes in 1876, with early teachers including George G. Hendricks, J.A. Jennings, J.T. Ray, and Sallie Crumpler. Pilgrim's Rest School #19 and Oak Grove School #2 began operating in 1891, and these schools later consolidated with Baxter School. In 1907, Willie and Annie Johnson donated one acre of land for education and the Woodmen of the World Fraternal Order, and a two-story frame building was constructed there. That Woodmen of the World building housed Baxter School for several years until a storm destroyed it in 1914. W.R. Thomas taught under trustees H.T. Reynolds, W.C. Frizzell, and W.E. Johnson. In 1915, W.E. and Annie Johnson sold another acre to the school trustees directly behind the original acre, and Camp No. 1408 of the Woodmen of the World laid the cornerstone for a new school building on Oct. 1, 1915. Frank J. Davis served as teacher from 1916 to 1928 and later became county school superintendent and served three terms as Henderson County Judge. Classes continued in the Baxter Schoolhouse until 1943, when the Baxter School District consolidated into the Athens School District. Afterward, the schoolhouse served the community as a place for meetings, family reunions, quilting bees, and voting. In 1951, Athens ISD sold the property to the Baxter Home Demonstration Club, and in 1997 the property was deeded to the Baxter Community Center. Repairs and modernizations occurred throughout its history, and the property continues to bring the people of Baxter together.