The Dutch Reformed Church in Roxbury was organized in 1802, and over the following years two church buildings were destroyed by wind and one by fire. In the 1890s Jay Gould promised aid for a stone church but died before any agreement had been made, so his six children took over the project, providing funds and sparing no expense on the building constructed in memory of their parents. Designed by H.J. Hardenburgh of New York City and completed in 1894, the impressive stone structure includes Tiffany windows in the apse and nave. The three apse windows depict the resurrection and show Mary, Christ, and an Angel, while the single nave window represents faith, hope, and charity. Windows in the east and west wings, designed by Maitland Armstrong, depict Christ blessing little children and a choir of angels.