The Lower Manor was the location of the manor house of the Lower Manor of the Van Rensselaer patroonship, built by Hendrick van Rensselaer in 1685, where tenants paid rents.