In February 1781, officials oversaw the construction of a picket fence around the prison yard to improve security. In 1802, Colonel Calvin Barber from Simsbury built the twelve-foot stone wall enclosing a 180 x 160 foot prison yard. The new wall expanded the prison yard and provided greater security, keeping prisoners in and unwanted visitors out. The brownstone used for the wall and other buildings most likely came from nearby quarries. An 1809 travel account described the prison walls rising gray upon the brow of the mountain. Both above and below ground, evidence remains of the people who resided there, including letters etched into the stone, many appearing to be initials, as reminders of the people who once worked and served time there. The foundation of the Centre Building has stood since 1790. In the 1830s, the Phoenix Mining Company removed the northeast corner of the perimeter wall to accommodate mining operations.
