African Americans were involved with Camp Ford from its beginning as a prison camp, with local slave labor building the stockade, four white officers of black troops detained there, and at least 27 black naval personnel held and apparently used as labor by their captors. The controversy over African Americans in the military became central to prisoner of war exchange policy during the Civil War, beginning in the Mississippi River theater rather than the east. Early in the conflict, pressure over retaliation led both sides to adopt formal exchange procedures through the Dix-Hill Cartel in June 1862, but as the North began recruiting African Americans in mid 1862, especially in Louisiana, Confederate resistance to recognizing black soldiers helped unravel the system. President Jefferson Davis suspended exchange of officers in early 1863, and the cartel was repudiated in July 1863, partly because the South refused to recognize the legitimacy of African American soldiers. Southern policy toward captured black troops and their white officers varied from execution to re-enslavement to more lenient treatment meant to encourage desertion. After the exchange system broke down, Trans-Mississippi commanders made their own arrangements, and General Nathaniel P. Banks required that exchanges make no distinction as to corps, meaning no discrimination against captured black troops or their white officers. In the late spring of 1864, opposing commanders created their own exchange agreement, and this issue was probably a primary reason for placing Richard Taylor's prison camp at Tyler, allowing prisoners to be kept in Texas and black troops or their white officers to be claimed as outside the relevant jurisdiction. After the February 1865 exchange, Lt. Frederick Crocker of the USS Cliffton reported that Confederate exchange commissioners had violated the agreement by refusing to release 27 black seamen who were held to labor at Camp Ford, and later Confederate accounts referred to 27 black laborers digging a trench to prevent tunneling, apparently the same men. Although Camp Ford never held large numbers of African American prisoners of war, the political and legal disputes surrounding them significantly shaped the camp's establishment, and while the small number of white officers initially faced threats of severe treatment, most were exchanged in the July 1864 transfer.