Settlers were drawn to the southern end of Holly because of the Shiawassee River, and in the early 1820's Ira C. Alger arrived to establish the first settlement. In 1843, Alger dammed the river at the border of modern day Rose and Holly townships, built a sawmill, and later erected a grinding mill on the west side of the Millpond, an important development for the small settlement because mills provided essential services during the frontier phase by grinding grain and processing lumber. By 1875, the mill had grown to employ ten men and produced 150 barrels of flour a day, shipped on a railroad spur connected to the Grand Trunk Railroad. "Holly Velvet" flour became nationally distributed as the Village of Holly grew with railroad development. Before the building burned down, the factory was running two shifts and shipping 300-400 barrels of flour daily, but after a fire that burned for days beginning on December 17, 1910, causing a $50,000 loss, the mill was not rebuilt.