T.J. Caldwell, a Tennessee native, and his wife Leticia built this house in 1860. Limestone for the walls came from the surrounding land, and cedar cut on the property was used for the foundation, rafters, and floor joists. Caldwell's slaves helped build the house. Their daughter Mary married Sven William Palm, and the couple bought the property from the Caldwell estate in 1892. The house remained in the family for more than 100 years. This two-story, five-bay, rough-cut limestone house has a center passage floor plan and a two-story wooden gallery, and it stood within the Caldwell Heights community, which included a school, cotton gin, and other rural homes.