First settled by Robert J. Scott on May 10, 1856, Milford became one of the oldest settlements in the Honey Lake Valley. In 1861, the first important flour mill in northeastern California was built there. In recognition of this, and of a sawmill, Joseph C. Wemple named the village. By 1882, the town had a sawmill, grist mill, hotel, store, blacksmith shop, butcher shop, post office, and school house.