Duffy's Peak was so prominent that it became Garza County's earliest bench mark, designated by peg-legged surveyor Jasper Hays as he marked the bounds of Llano Ranch in 1877-1878, the first property occupied by settlers in the county. Hays began at a corner established for the Houston and Great Northern Railroad Company near the White River in Crosby County and shot west toward the peak. Tradition holds that a member of Hays' crew died and was buried near the peak, which was afterward named for him. Other local heights were named for natural wonders, including The Ice Cream Cones, Cow Head Mesa, Indian Head Point, and The Chimneys, while other notable uplands included Two Bush Hill, The Devil's Breakfast Table, Needlepoint Peak, and Buffalo Point. Two balanced rocks known to early settlers later disappeared through wind and weather erosion. Like most of the scenic formations, Duffy's Peak is composed of sandstone, clay, and sand. It continued to figure in local land transactions because Hays' original notes, unearthed from Mason jars he had buried on the peak, formed the basis for later surveys, including those of A. L. Marhoff in 1906 that established boundaries for C.W. Post's farm colony.