In 1912 Betty's Bay became a formal whaling station running until the 1930s. A significant part of the history of Betty's Bay came in 1913 when the whaling station at Stony Point was established. Frank Cook initiated the project and started off by leasing approximately 24 hectares of land from the Walsh brothers. In the 17 years that the whaling station was operating, two whaling boats hunted the whales off the coast of the Hangklip-Kleinmond area. The whale of choice was mostly the Southern Right Whales because they are one of the easiest whales to hunt and catch, a fact that lent them their name as they were the “right” whale to hunt. They were easier to catch compared to other whales because they venture close to the shore, they are naturally inquisitive so they often swim up to boats and they float when they are killed. Remains of the whaling station can still be seen at Stony Point. The area is named after Betty Youlden, daughter of the first developer of the area.
Betty's Bay contains the well known Harold Porter National Botanical Garden as well as an African penguin colony at Stoney Point. The botanical garden of Harold Porter, which stretches from the mountain-top down to the sea, is known as ‘little Kirstenbosch’ and contains some of the best examples of local fynbos, including proteas, restios and over 50 species of ericas.
Due to its isolated nature, it became a popular retreat for outlaws and the remnants of the Khoisan tribes. The Khoisan tribes had depleted in numbers due to measles but the population gradually grew in the area and runaway slaves and criminals also began to call the area home.