Members of the Fortson family came to Texas from Mississippi and settled in Navarro County in the 1840s. Civil War veteran James T. Fortson married Ida Clayton, daughter of another pioneer area family, in 1867, and they moved to the Rice area in 1872. Their son, Joseph Benjamin (Joe B.), and his wife Lyda built their home at this site in 1904. Joe B. and his brother John acquired large landholdings in the area for farming and ranching, established a cotton gin operation and a mercantile, opened the First State Bank of Rice, and began levee improvement districts to aid local farming. Joe B. also served on the school board, and the family is remembered for its lasting contributions to the area's history.