The Calvary Baptist Church and Annex, formerly St. Paul's Episcopal Church, were built during Ossining's 19th century industrial boom. As the population grew, local Episcopalians organized in 1833, bought land in 1834 from Richard Austin for $700, and began construction that same year. The church, built of Sing Sing marble, was consecrated in 1836. A spire was added to the bell tower in 1870 and later removed in 1951. A chancel and other elements added over the centuries did not detract from its status as the best preserved early Gothic Revival church in Westchester County, and it is also the oldest surviving religious structure in Ossining. In 1864 a rectory annex was added in the Gothic Revival style, with pointed arches and symmetry complementing the church. The annex was surrounded by an orchard used recreationally by the congregation. Its interior remains well preserved, with its original staircase and decorative ceilings, door surrounds, and window surrounds intact. Together, the church and annex form an architecturally significant pair of structures and stand as elegant reminders of Ossining's 19th century economic prosperity.