Bow Valley rock climber taking navigation tools to next level through augmented reality

Tied in, chalked up and with Apple’s Vision Pro goggles snug to his face, Pearman started up the Kahl Wall on Mount Yamnuska, a classic Rockies line where finding your next hold isn’t exactly obvious.

The longtime climber and computer science masters student at the University of Calgary could see more than just the jagged limestone route in front of him through the bug-eyed lenses.

“You can actually see your position on this 3D terrain map of the cliff,” said Pearman, of the digital world projected onto the wall.

“You literally see this clairvoyant route line that is weaving its way up the wall where you have to go,” he said. “Even the holds light up.”

It was the first test drive on real rock for Pearman’s experimental app, RockXR, that uses spatial data to guide climbers with “on-the-wall navigation.”

“It’s the most immersive way to navigate possible,” he said.

Pearman, like most climbers, has had his fair share of epics, getting off route on a climb and having to course correct on tricky vertical terrain.

And when the climber dreamed up the first iteration of the digital wayfinding tool with an app called ThreeTopo, it was in hopes of “never questioning if he was on the right route again.”

“You always have a moment of doubt of, ‘Damn. Is this the right one, or am I on some heinous random project that’s half-finished that’s about to end halfway up the wall?’” Pearman said.

“It happens to everybody.”

Using a drone to create a 3D digital copy of the trad route, Red Shirt, on Yamnuska, Pearman created ThreeTopo, but it came with its limitations, he said, noting the app requires use of at least one hand while you’re on the climb and it can’t actually determine where you are on the wall.

“GPS doesn’t work on vertical terrain,” he said.

“It’s not like using Google Maps.”

Back to the drawing board, Pearman created the augmented reality app RockXR, a continuation of ThreeTopo, that he describes as the “Google Maps experience for rock climbing.”

“We thought, let’s look into the future and assume that augmented reality headsets keep getting better and better over time. Eventually they’re gonna be really small. It’s gonna be like wearing a small pair of glasses,” he said.

Out of the lab and in the brisk Rockies air, Pearman and his friends, through AR glasses, could see floating icons indicating where holds, anchors and gear placements were on the trad route.

“As you move your hands around, you can see those little digital hands moving around,” Pearman added.

If a climber wants to get rid of the holographic images in front of them, they can switch to a voice mode, Pearman says, where an AI assistant turns the data into live beta.

“You can ask questions like, ‘How far away am I from the anchor?’  or, ‘Am I on route?’ or ‘Where’s the next piece of gear?’ And the AI system will basically look at where you are in the … digital shadow,” Pearman described.

While there are existing apps that use 3D topos and augmented reality tech, like The Climbing Guide and Red Point, Pearman says ThreeTopo still introduces something new.

“ThreeTopo was unique because it was very specifically for multi-pitch trad climbing,” he said.

RockXR stirs debate

When news of Pearman’s RockXR experimentations reached the broader climbing community, not everyone was on board.

“RockXR definitely stirred more of a debate, because it’s so weird,” he said. “People have said to me, ‘This is gonna result in death … the fact that people are using augmented reality to navigate while rock climbing.’”

But Pearman argues that the technology is only adding another option for the challenge of climbing but still gives climbers the choice of whether to use it or not.

“Climbing goes through a lot of phases, and people debate about ethics with essentially everything – everything to do with access to, should you bolt the routes, or should you not bolt the routes?” he said.

“That’s the important thing with probably any development of technology; is doing this going to reduce the sport or expand it?”

Canmore climber and guide, Greg Barrett, says he sees both the benefits and drawbacks of the high-tech tools.

“Whether these advancements are good or bad depends on what you’re trying to get out of your time in the mountains,” said Barrett.

For him, a real adventure in the alpine looks like “unplugging” from a lot of the gadgets and technology often vying for his attention.

“What I really like is feeling self-reliant and confident out there and confronting a challenge with a partner and working through it together,” Barrett said. “I’m not just there to climb the thing that’s been climbed.  I’m there to go have an adventure.

“Part of that adventure is that navigation and that planning and that trying things and that being wrong and correcting,” he added.

On the other hand, Barrett acknowledges apps like Pearman’s can reduce some risk, especially for newer climbers.

“Having these tools could keep people in the right place on the right climb and that can be a lot safer,” he said.

“If you have an app that’s letting you just look at the wall and see where you’re supposed to go then that can just make your day more fun and less time spent faffing about.”

But that shouldn’t come at the price of building foundational navigation skills in the mountains.

“If you do want to go somewhere that doesn’t have that AR guiding you, but you’ve spent your whole time with these goggles on having everything hand-fed, you don’t develop the skills to look up at the rock and go, ‘well, what would make sense?” he said, noting that the same risk exists with relying entirely on a phone GPS.

“You kinda need to make some wrong turns because they’ll be very memorable.”

But unlike the longstanding debate over whether to bolt routes or climb traditional, where climbers place and remove their own gear on a route, the new technology still presents a choice, Barrett says.

“If you go in and take this classic route and add a bunch of bolts to it, you’ve changed it forever now,” he said. “It fundamentally changes things.”

“In the case of … technology, it’s nice that one doesn’t happen at the expense of the other,” he said.

“If someone is using these tools and you’re not, it doesn’t detract from anyone else’s experience.”

AR climbing still a long way off

While discussions of augmented reality in climbing can make it feel like robots for belay partners are up next, Pearman says concepts like RockXR aren’t knocking at the door just yet.

“It’ll just take a long time, I think, for that to reach a super stable, practical point,” he said. “It probably wouldn’t look exactly like RockXR. It would have to be a simplified version of it.”

Barrett feels that another big hurdle is simply the amount of Rockies routes that exist.

“The challenge that we have today with this information isn’t that we don’t have a way to present it to people, it’s just that there are perhaps tens of thousands of climbs in the Rockies, I would guess, … and there’s no one central resource for them,” he said, adding that he often consults multiple guidebooks, Facebook, Mountain Project and other apps when he is planning to attempt a new route – and they can all contain different levels of accuracy.

“If we can’t get it together with the apps and the books, then we’re definitely not going to get it together with the drones and the headsets.”

“I don’t think the community needs to stress about it having a big impact in the foreseeable future, even though it is super cool,” Barrett added.

While the climbing metaverse isn’t here just yet, Pearman is betting on 3D topo navigation becoming more standard.

“I think every multi-pitch climb, probably eventually every crag, in the world will be a 3D topo,” he said.

“It’s a much more natural way to understand where you’re trying to go because it’s spatial. Trying to collapse hundreds of metres of extremely complex 3D terrain into a two-dimensional drawing with a pencil is pretty hard to do.”

The Local Journalism Initiative is funded by the Government of Canada. This position covers Îyârhe (Stoney) Nakoda First Nation and Kananaskis Country.