Today, the narrow channel of rushing water under Tillicum bridge is little more than a barrier to most paddlers, but it was once the place to be on hot summer days.
Daredevils dove off the bridge, much to the authorities' displeasure, and strong swimmers took their chances riding the current at what's known as the reverse falls.
Metres away, the two-storey Free Bathing House offered changing facilities and showers. Built in 1911, it recorded 12,000 swimmers during its fifth summer of operations, according to Dennis Minaker, who researched the history for his book The Gorge of Summers Gone.
Kids from Oak Bay would take two streetcars to get to the Gorge, he said. They'd pack a lunch and stay all day. To keep up with demand, a streetcar arrived every 10 minutes.
In its heyday, the Gorge was a playground, with established swimming holes dotting the shoreline on both sides. There were rope swings, diving boards and slides set up on floating docks.
Further up the waterway, a pay bathing house accepted swimmers for a fee off the shore of what's now known as Esquimalt Gorge Park.
Until the Crystal Pool opened in 1925, the Gorge was the only place for swimming lessons. Six swimming clubs hosted water sports and swimming championships that attracted top swimmers from around the Pacific Northwest.
Competitions often included some silliness. Minaker's photo album includes pictures of barrel races and front-stroke competitions where racers are holding umbrellas overhead. During a backstroke race, competitors had to swim with a lit candle in their mouth, Minaker said.
On a warm evening, the beach would be full of swimmers; you'd hear the live dance band and the music of the merry-go-round, and see people touring the waters by boat, he said. "All of that would have been so appealing."
It all started to change in the 1930s and 40s, when pollution levels in the Gorge made swimming less appealing.
For his book, Minaker interviewed hundreds of people with fond memories of summers at the Gorge.
"I started swimming there in the mid-90s because all these old timers said how much fun they had as children," he said.
While dozens of people now join him, Minaker was one of the first to venture out after the signs came down warning against swimming.
"I have no reservations about swimming in the Gorge."
rholmen@vicnews.com