Stoughton's first schoolhouse was built on this site in 1768. By 1799, the schoolhouse had been moved to Central St., and Benjamin and Abram Capen built a tavern here that hosted sings by the Old Stoughton Musical Society and meetings of the Masonic Rising Star Lodge. The Capen Tavern was later demolished after the Civil War, and the lot remained vacant until construction of the public library in 1903. Lucius Clapp donated $25,000 toward construction of the library, which now bears his name as the home of the Stoughton Historical Society. Stately elms graced the site in the mid-Twentieth Century until they succumbed to Dutch Elm disease.