Named for an English martyr and saint, the parish was organized in 1842. The first church, built in 1844, stood at the southwest corner of God’s Acre, where the first grave was opened that year. The edifice was dedicated by Bishop Jackson Kemper in 1866, and the crenelated tower and bell were added in 1875. It was patterned after the village church in Peasmarsh, Sussex County, England, the home church of some of the founders, and was regarded as a finer example of the English Gothic-styled parish church of the period. The guild hall was erected in 1904.