The historic Federal Hill / Evergreen House was built in stages, beginning circa 1780 with a log/frame house built by Edward Grimes. Circa 1822, George Winter added onto the front porch of the Grimes' house, which faced southward toward the Barn, building the home's two story stone addition. In the 1870s, Winfield Trimble tore down most of the original Grimes' House and replaced it with a two story frame addition, creating a six bedroom mansion.