The mansion was built by Isaac Royall, who came here from Antigua with his slaves in 1737. His son, Isaac Royall, a loyalist, founded at Harvard the oldest law professorship in the United States. It also served as the headquarters of General John Stark during the Siege of Boston.