Built about 1698 by Jacob Leonard, this house was sold to Rev. Allen, minister of the First Parish Church, who, under pressure for his Tory sympathies, sold the property to Col. Josiah Edson, a Tory who occupied it until 1775, when it was confiscated by the Continental Congress. Prior to the Civil War it was a station on the Underground, helping runaway slaves escape to Canada.