Anthony Frazier was born in Georgia in the mid-1820's and spent most of his early life enslaved on the Delk Plantation at Rock Springs. After Delk freed him, Frazier and Robards went to South Carolina and enlisted in the Union Army for three years, after which he would receive $100. Frazier served in Company K, 21st United States Black Infantry, and when Charleston surrendered on February 18, 1865, his regiment was the first to march in. While stationed in Charleston, he met his future wife, Mary Ward. On the day they were to be married, he was ordered to board a train for Savannah and continue to his garrison in Jacksonville. He was honorably discharged on April 25, 1866, but almost a year passed before Mary could move to Jacksonville and they could finally marry. After Anthony died, many members of the community wrote letters supporting Mary, his wife, in receiving his pension.