Early in the war, Confederate authorities feared a Union advance up the James River from Fort Monroe and, as a precaution, built waterfront artillery batteries along the river, including in Isle of Wight County, with nearby infantry regiments posted to protect them. The war reached the grounds of St. Luke's Church when Col. W. Dorsey Pender's 3rd North Carolina Volunteers, later reorganized as the 13th North Carolina Infantry, camped there as part of Gen. John Pemberton's brigade and supported a North Carolina battery at the mouth of Jones Creek north of the church. In July 1861, Pender wrote his wife that he called the place Camp Ruffin, that the camp's location and appearance were anything but agreeable in looks or comfort, and that three camps stretched over about three and a half miles. In November 1861, Union Assistant Adjutant General William Whipple reported that escaped slaves told him of a large Confederate force on the grounds of the old brick church near Smithfield. In the spring of 1862, Confederate forces crossed the river to oppose Union Gen. George McClellan's advance up the Peninsula toward Richmond, and after the camp was vacated, Harper's Weekly reported that the church was left entirely despoiled of everything movable or destructible.