Bay Creek Baptist Church was established in 1896 by Rev. Joseph Lewis Bryant, a member of the Mississippi House of Representatives from 1908 to 1912, who also served as the congregation's first pastor. Its first sanctuary was built in the early 1900s and was replaced in 1938 with a sanctuary built with local heart pine lumber. The cemetery was established in 1903 and contains the remains of two Confederate soldiers, Elijah Burt and J. F. McGrew, and veterans from World War II and Korea.