The LeRoy House was erected prior to 1812 by Egbert Benson, the first land agent of LeRoy. In 1821 it was enlarged and occupied by Jacob LeRoy, the second land agent and son of Herman, for whom the town was named.