The community was originally named Ambersonville after one of the early families that settled there. A post office was established in 1842 as Liberty, and the name changed to Spring Garden in 1844. Rail service through the town began in 1870 with the Selma, Rome and Dalton Railroad. The line later became part of the Southern Railway and was abandoned in the 1970's.