Samuel Kidder Whiting (1817-1882) invested in sailing vessels and belonged to a large mercantile family with branches in Castine and Ellsworth. In 1822, his father, Jarvis Whiting, built a tavern on the site to serve travelers. The tavern was later used as a home and moved to the neighboring lot. The corner lot then became the site of the Unitarian Church, designed by S.S. Woodcock Architects of Boston and built by I. & O.W. Kent. The church was dedicated on August 28, 1867. In September 1875, the Unitarian Social and Benevolent Society raised funds to add a vestry to the church. After falling into disrepair, the church was torn down in 1971.