Millwood developed around two late-18th-century gristmills and Nathaniel Burwell’s Carter Hall plantation, one of the preeminent estates in the area. The Burwell-Morgan Mill in the center of the village was a commercial gristmill, while the Carter Hall Mill served the plantation. In 1865, Confederate Col. John S. Mosby discussed terms of surrender in the J.H. Clarke house and tavern (1842) located across the road. After the Civil War, Millwood included a community of freed blacks with a school and several churches. The village retains many late-19th-and early-20th-century stores and dwellings.