David Goss, coming from Massachusetts in 1804, took up land on the Holland Purchase at the first permanent settlement in the Town of Pembroke. His dwelling, which he converted into a tavern, was the first public house in the township. To the west 3.25 miles on the Old Buffalo Road stood another tavern. Legend holds that a would-be guest, identifying herself as the "Dutchess of Sligo," so riled "Inn-Keeper Blogett" that he retorted, "I am the Earl of Pembroke" and refused her lodging. His tavern thereafter became known as the "Earl of Pembroke's Castle." Richville became Pembroke.