The Civil War made civilian life in Union County difficult and dangerous as most residents opposed secession and were staunch Unionists, with men joining Federal regiments such as the 1st Tennessee Infantry and the 2nd and 9th Tennessee Cavalry, while others sided with the Confederacy. These divided loyalties splintered families and caused violence among neighbors. Union County's proximity to Cumberland Gap, which changed hands several times, led to two military engagements there. On June 30, 1862, Union Col. Robert K. Byrd's 1st Tennessee Infantry, with two companies of the 3rd Kentucky Infantry, was foraging along Powell River west of Maynardville when several companies of the Confederate 1st Georgia and 1st Tennessee Cavalry regiments attacked; five Federals were captured, but Byrd's men captured one Confederate. Union Gen. Ambrose E. Burnside secured Cumberland Gap and occupied the area in September 1863. On December 1-2, Col. Felix Graham's 5th Indiana Cavalry skirmished with Confederate forces at Walker's Ford a dozen miles northeast on the Clinch River and at Blaine's Crossroads about eight miles southeast. Federal occupation and Confederate raids wreaked havoc on farms and market patterns, and foragers stripped crops, livestock, and fences from the landscape. Slaves and free blacks fled to the Federals in such numbers that the county's black population had fallen 20 percent by the end of the war. After hostilities ceased, former slave Samuel Smith established the county's only school and church for blacks on 110 acres that he purchased.