Built in 1866 as a one-room schoolhouse for Catholic boys, the building became the rectory for Father F.X. Blanchet in 1870. An addition was built in 1891. When St. Joseph's Catholic Church became a mission of Medford in 1908, the rectory became a private residence. Local resident Robbie Collins saved the building in 1967, and it is presently preserved and maintained by The Friends of St. Josephs.