Hendrick I. Lott inherited his father's property, including enslaved persons, but no slaves were documented there in the 1800 Census, while four free persons of color lived on the property after being freed more than two decades before slavery was officially abolished in New York State in 1827. Manumission slips record some of the people Hendrick freed, and family history holds that the Lotts supported the abolitionist movement. Family oral history recounts a closet within a closet on the second floor of the Lott House where freedom seekers were hidden as part of the Underground Railroad, and newspaper pasted to the wall there is dated June 10, 1863. The story gains credibility from these details and from the house's location beside Gerritsen's Creek, a tidal inlet accessible to boats coming up the eastern shoreline, and near Flatbush Road and Kings Highway, which offered a relatively direct route to Weeksville, the free African American community in present-day Crown Heights.