Monterey, the seat of Highland County government, served as headquarters for both Confederate and Union forces at different times during the Civil War. Confederate Gen. Henry R. Jackson, who led a brigade under Gen. Robert E. Lee during the First Campaign of 1861, established his headquarters there. In the winter of 1861-1862, skirmishes took place across the county as the no-man's-land between the armies shifted from the Allegheny Mountains on today's state line to West View in Augusta County. On April 6, 1862, Union Gen. Robert H. Milroy and his forces occupied Monterey and used the Osborne Wilson-Fenn House on Main Street as headquarters. The town remained in Federal hands until after the Battle of McDowell on May 8, 1862, near the beginning of Confederate Gen. Thomas J. "Stonewall" Jackson's Shenandoah Valley Campaign. Many western Virginians were ambivalent about choosing sides, and Highland County's government waited until June 1862, after Jackson's success in driving the Federals from the Shenandoah Valley, to take the oath of allegiance to the Confederacy.