Built in the style of a Scottish stone croft, this house was erected by James Whitehill, a prosperous farmer and one of many immigrants from Inchinnan Parish, Scotland, who settled Ryegate under the sponsorship of the Scotch American Company of Farmers. Whitehill purchased 600 acres, known as the Witherspoon Tract, from James Witherspoon, a signer of the Declaration of Independence who owned the entire township. He was also a founder of the Ryegate Reformed Presbyterian Church. The house is one of the oldest surviving in Ryegate, served as the town's first library, and its surrounding acreage remains owned by the Whitehill family.