By 1842 this house stood on Pawleys Island and was owned by Robert Nesbit (1799-1848), a native of Scotland, a rice planter in this area, and the owner of nearby Caledonia plantation. The house remained in the Nesbit family until after the death of Ralph Nesbit in 1938, when it was sold to Dr. Charles Norburn of Asheville, North Carolina.