In 1704, when Maryland's governor ordered the chapel door locked and the legislature passed the Act to Prevent the Growth of Popery, Catholics in the colony were barred from holding office and voting, were double taxed, and were allowed to worship only in the privacy of their homes, while Jesuit priests were outlawed but continued to minister to them. Despite these restrictions, the Catholic population grew, and by 1776 Maryland had the largest number of Catholics of any of the thirteen colonies. Charles Carroll of Maryland became the only Catholic to sign the Declaration of Independence and gave significant financial support to the Revolutionary War effort. After the Revolution, religious freedom returned, and Maryland's Catholic population became the cornerstone of the Catholic Church in the new United States. John Carroll, a Jesuit and cousin of Charles Carroll, became the first bishop in the United States, with Baltimore as the first diocese, and in 1806 he began building a cathedral there that directly descended from the small brick chapel built at St. Mary's City in the 1660s. In this sense, St. Mary's is the birthplace of the modern Roman Catholic Church in the United States.