Federal forces under Brig. Gen. John Buford, Brig. Gen. David Gregg, and Col. Thomas Devin, whose command held the area around St. James Church, had almost encircled the Confederates, though none of the Union commanders had planned it so. Unknown to the Federals, Gen. Robert E. Lee was visiting the Barbour House, called Beauregard, to scan the fighting on Fleetwood Hill, and his capture would have been a devastating blow to the Confederacy. By late morning, Confederate Brig. Gen. W.H.F. "Rooney" Lee pulled his forces out of the fight with Buford's troops over the stone wall separating the Cunningham and Green farms, and they fell back in a fighting retreat to Yew Ridge and northern Fleetwood Hill. Battlefield terrain played a crucial role in the battle's progress, as the steep, rolling topography exhausted the mounts of the pursuing Federals. Toward the end of the afternoon, Buford calculated that his men and their mounts were too worn out to continue the fight. Fresh Confederates, the tardy brigade of Col. Thomas Munford, menaced Buford's retreat route across Beverly Ford, and the Federal general pulled back to the ford, yielding the field to the Confederate horsemen.
