The Montgomery House was built in 1771 by Palantine settler Hans Smith and was enlarged in 1791 by Johannes Miller, a promoter of the Cochecton Pike, Montgomery Academy, and the County Agricultural Society.