Successive generations of the Robinson family shaped Rokeby's history. Thomas Richardson Robinson and Jemima Fish Robinson, raised as Quakers in prosperous Newport and Portsmouth, Rhode Island families, moved to Vermont in 1792 and settled at Rokeby a year later, where Thomas established saw, grist, and fulling mills on Lewis Creek and in 1810 purchased Merino sheep from Spain. Their son Rowland Thomas Robinson, born at Rokeby, married Rachel Gilpin in 1820 after meeting her at school in Dutchess County, New York, and together they returned to Rokeby, managed the sheep farm and mills, became radical abolitionists, worked in antislavery societies locally and nationally, kept their home free of slave-made goods, and supported freedom seekers there. Their son George Gilpin Robinson managed the family farm, helped convert it from wool to dairy with his brother Rowland, served for a time as Ferrisburgh Town Clerk, and in 1893 added a one-up, one-down addition to the south side of the home for the Town Clerk's Office with a bedroom above. Rowland Evans Robinson, the youngest of four children, trained as an engraver in New York City before returning to run the farm with George, married Ann Stevens in 1870, and after failing eyesight ended his art, turned with Ann's help to writing, gaining renown for stories of life in Danvis and for celebrating Vermont's natural beauty as a naturalist and conservationist. Their daughter Rachael Robinson Elmer became a distinguished commercial artist and book illustrator, studied at the Art Students League in New York City, settled in New York after marrying Robert Elmer, published notable fine-art postcards of New York scenes in 1914 and 1916, and died during the flu pandemic in 1919 at age 40. Another child, Rowland Thomas Robinson, known as Rowlie, inherited the farm, ran it with his wife Elizabeth Donoway Robinson, and as Vermont farms declined, they began taking in tourist boarders in the 1920s to supplement income; because they had no children, Elizabeth left the property and its contents in 1961 to be operated as a museum. The youngest child, Mary Robinson Perkins, studied at Goddard Seminary and the University of Vermont and combined the family's love of nature and artistic talent in work as a botanical artist before marrying Llewellyn Perkins and raising two children.