This house was built circa 1810 for Milton Maxcy (1782-1817), who came from Massachusetts in 1804. Maxcy and his brother Virgil, who founded a school for young men in Beaufort, later taught at Beaufort College. In the 1850's Edmund Rhett (1808-1863), a lawyer, planter, state representative, and state senator, bought the house and extensively remodeled it in the Greek Revival style with an elaborate two-story portico. Along with his brother Robert Barnwell Rhett (1800-1876), a lawyer, state representative, state attorney general, U.S. congressman, and senator, Edmund Rhett was an outspoken champion of state rights and Southern nationalism from the 1830's to the Civil War. Long known as "Secession House," it was the scene of many informal discussions and formal meetings during the 1850's by the Rhetts and their allies advocating secession and Southern independence.