The Knox family home had belonged to the family since colonial days. James Knox, grandfather of President James Knox Polk, lived there. The house stood 4 miles west.