Built in 1807 by Nathaniel Winsor, Jr. and Hannah Loring Winsor, this grand Federal house on the corner of Washington and Mattakeeset Streets is believed to be Duxbury's only three-story Federal and stood at the center of the town's business community, overlooking the waterfront and the Winsor family's shipbuilding wharves. Federal architecture, popular between 1780 and 1830, rose in Duxbury during a period of prosperity, especially along Washington Street, and many such houses survived because later owners lacked the money to replace them. Nathaniel Winsor, Jr., a carver of figureheads, inherited a thriving mercantile enterprise begun by his grandfather Samuel Winsor, and the family built at least 40 sailing vessels in Duxbury, including large brigs that traded in ports around the world. After Duxbury shipbuilding declined in the 1840s, the family business moved to Boston, where the Winsors acquired steamships and operated one of the first regular clipper ship lines from Boston to San Francisco. After the deaths of Nathaniel Winsor, Jr. and Hannah Loring, the house remained in the family for the rest of the 19th century, and in the 20th century it passed through a series of owners and often operated as an inn for the town's growing summer population. Elizabeth Winsor Sampson, born in the house in 1809, was among the Duxbury women who went to sea with their husbands on voyages around the world; after marrying Capt. Erastus Sampson in 1830, she sailed with him aboard the Duxbury-built ship Coriolanus and traveled on at least one other voyage in 1845. The Sampson family owned the house from 1835 to 1893, and in 1997 the Duxbury Rural and Historical Society purchased it through a community fundraising effort.