Bloomsbury Presbyterian Church was founded in 1857 as a daughter church of the Old Greenwich Church to the west in Warren County, and its building was erected in 1858.