In 1786 John Saxe, Johannes Sachse, a Loyalist from Rhinebeck, N.Y, built the area's first gristmill here. His sons added a sawmill, potashery, general store, post office, and tavern. They incorporated the town of Highgate in this house in 1805 and served in numerous offices: Matthew as Highgate's first elected Town Clerk, Conrad as Captain of the militia during the War of 1812, and Peter as a member of the Vermont General Assembly and Franklin County Judge. Peter's son, John Godfrey Saxe, born here in 1816, ran twice for Governor but is best remembered for his poetry.