The house stands on its original site overlooking the Raritan River on land that was once part of an 834 acre plantation purchased in 1697 by Michael Van Veghten. Derrick Van Veghten, born there in 1699, became a prosperous farmer, a respected member of the community, and a staunch patriot. His house served as headquarters for Quartermaster Nathanael Greene during the 1778-1779 Middlebrook Encampment, and units of the Continental Army were encamped on his land. While living there, Greene wrote to a friend about "a pretty little frisk" held in the house, where General Washington danced with Mrs. Greene for "upwards of three hours without seting [sic] down."