Buffum Inn was built in 1828 by Richard Buffum, the first settler in 1810, who built the first grist mill in 1814, giving the area the name of Buffum's Mills.