Son of oyster king Thomas Downing, George launched his own New York catering business. In 1854, he built the Sea Girt Hotel in Newport, Rhode Island, which featured a restaurant and confectionery. After the hotel was lost to arson, Downing managed the House Refectory for 12 years.