A small wooden farmhouse once stood on this property, where James Fenimore Cooper lived for four years in the mid-1810s with his wife, Susan, soon after their marriage. After 1829, future U.S. Supreme Court Justice Samuel Nelson enlarged the house during his nine-year residency. In the early 1930s, Edward Severin Clark made a more drastic change by replacing the old farmhouse with a Colonial Revival mansion later known as Fenimore House; although early plans attached the farmhouse as a southern wing, it was replaced by a new wing before the house was completed in 1933. Clark, who had acquired the Greek Revival farmhouse through his grandfather's will and lived there for several decades before deciding to build a new home, died in 1933, and in 1944 his brother, Stephen C. Clark, offered the house to the New York State Historical Association. A new wing built in 1995, together with extensive interior renovations, transformed Clark's home into today's Fenimore Art Museum.