The Smith-Robinson House is one of four remaining Sing Sing marble buildings in the Village, built of Hudson River marble, technically a dolomitic limestone, quarried at Sing Sing Prison and popular as a nineteenth-century building material. In the 1830s, James Robinson purchased the Smith property, which extended to Route 9 and south to the State Farm property near Nelson Park, and built this mansion overlooking the Hudson. Financial difficulties forced him to subdivide the property and sell the lots. In the twentieth century, the building was part of the Catholic Community club recreational facility. Designer and artist Vera Neumann, known for her signature scarves and textiles, purchased the property in the 1950s. The house has one of the best Greek Revival interiors in Ossining.