Chartered on December 12, 1889, Ashley Lodge No. 681, A. F. & A. M., was the seventh Masonic lodge organized in Fannin County. Its first official meeting occurred in January 1890 in a two-story frame building shared with the Baptist church. In 1912 the lodge purchased a separate building in downtown Bailey, reserving the second floor for lodge meetings and leasing the first floor to a variety of retail businesses. That structure burned in 1950, and the members met in the Grove Hill Lodge in Leonard until a new building was completed in 1952.