Richard Williams, born in Tennessee in 1830 as the son of Judge Arthur Williams, came to Johnson County in the 1850s as a surveyor employed by the U.S. government. During the 1850s and early 1860s, he worked as a wagonmaster, escorting wagon trains from Kansas City to Santa Fe and other western destinations. Sometime in the 1850s, he hired the Garrett brothers, stone masons, to build his two-story limestone house on a site allotted in 1857 to Shawnee Indian John Pumpkin. Williams family tradition held that the house was built in 1853, although it was most likely built between 1857 and 1860. The house became known as the Wagonmaster's House at the southeast corner of Shawnee Mission Parkway and Nieman Road. It survived about 100 years before being dismantled in 1959, and its limestone rocks were removed to Shawnee Mission Park for possible future reconstruction in Old Shawnee Town.