The cave was formed as flowing water dissolved limestone, leaving a large natural chimney in the ceiling at the back where water seeped into the rock from above. As the limestone dissolved, calcite was deposited to form flowstone, including stalactites and stalagmites that developed slowly over hundreds of years, though early cave explorers removed most of these formations. During the American Revolution, a British loyalist named Jacob Salisbury led raids on patriot farms in the valley below, then escaped with his raiders to this well-hidden cave after burning and looting. Smoke rising through the cave’s natural chimney eventually revealed the hideout to a search party, and the story is told that Salisbury was captured, tried, and hanged there, after which the cave became known as the Tory House.