Self-proclaimed “mad scientist” Bertrand Nepveu wanted to take a sip of beer but couldn’t see the bottle while wearing his start-up’s virtual reality headset.
The dreadlocked engineer therefore sought a solution to not be able to see real objects while being immersed in a virtual world. His company Vrvana repurposed the cameras in its “Totem” prototype, which was designed to detect a user’s position, to allow users to see their surroundings as well.
This breakthrough led Apple to acquire Vrvana for $30 million in 2017. The acquisition is just one of twelve virtual and augmented reality purchases the iPhone maker has made over the past six months. years.
Apple’s own headset, the long-awaited culmination of all those deals and a huge investment in research and development, is shaping up to be its most important new product since the iPhone.
Apple’s Worldwide Developers Conference in San Francisco on June 6 could be the $2 billion company’s last big gathering before unveiling the headphones as soon as this fall, giving it one last chance to rally fans and chart the course. way to a new generation of computing hardware.
Little is known about Apple’s new product, but Vrvana’s “pass through” video system should be a key feature. The idea is to turn the closed world of VR headsets upside down, allowing the device to allow users to also see the physical world around them overlaid with digital images.
“We redid everything to merge AR and VR, for the first time, in a single device,” said Nepveu, who declined to discuss his work at Apple before leaving last year.
Such breakthroughs have led to the belief that Apple could once again launch a device that could transform the technology, media and entertainment industries as it has done before with the iPod, iPhone and iPad.
“I believe there is a way you [won’t] need your phone and [augmented reality] eyewear will evolve into the next computing platform,” said Cristiano Amon, chief executive of Qualcomm, a chip supplier to the world’s largest device makers including Apple, Samsung and Meta, Facebook’s parent company.
While few analysts expect AR to become mainstream overnight, Citi researchers expect 1 billion people – roughly the number of iPhone users today – are wearing helmets by 2030, supporting a market worth up to $2 billion in revenue.
To achieve this lofty goal, Apple and its Silicon Valley rivals are racing to solve technical challenges. For many, the way virtual reality cuts users off from the world is clumsy and antisocial. Some users have complained of feeling vulnerable or stupid while wearing bulky headsets. Enough gamers get injured for “best VR crashes” to proliferate on YouTube.
These issues have led to existing headsets made by groups such as Meta remaining a niche product with few regular users other than gamers.
Timoni West, vice president of emerging tools at Unity, a video game software company that works closely with Apple and iPhone app developers, calls it “the biggest problem” in virtual reality.
Most people have little interest in wearing something West likens to a snorkel mask. “It can still be a bit scary,” they said. “And for a class of people who want to be able to see everyone in space with them, then that’s kind of a no-start. »
The VR market remains tiny compared to PCs and smartphones, but is currently dominated by Meta. Sales of its Quest 2 headset reached around 10 million units last year, giving Meta 78% of the VR market, according to research house IDC. This gives Meta a leg up on Apple in understanding what consumers want and attracting app developers to its platform.
Meta now has 17,000 employees working on “metaverse” technology, with a budget of around $3 billion per quarter. Mark Zuckerberg has said his high-end ‘Cambria’ headset – another hybrid VR/AR device with pass-through video similar to those rumored for Apple’s first device – will launch later this year.
This investment makes it unlikely that Apple’s first headset will do to the Quest what the iPhone did to the BlackBerry anytime soon.
“We are far from [Apple chief executive] Tim Cook took the stage and said, “It’s the device that replaces your smartphone,” says Sam Cole, CEO of FitXR, a virtual reality fitness app.
Multiple innovations are needed in areas as diverse as battery power, user interface, and physical comfort for VR and AR headsets to transition from niche gaming applications to the daily routine of tens of millions of people.
When Apple created the iPhone, its biggest breakthrough was replacing a physical keyboard with a multi-touch screen.
Matt Tullis, senior executive at Tencent-backed startup UltraLeap, predicts that using natural hand gestures, rather than game controllers or gamepads, will be key to integrating augmented reality. “Manual monitoring will consist of [AR] what multi-touch was for the iPhone,” he said.
Tipatat Chennavasin, a VR investor at the Venture Reality Fund, said he expects Apple’s first headset to be an expensive device aimed at professional designers and developers, the same customers who adopted the Macintosh a while ago. decades. “It’s not the iPhone time, it’s the Mac time,” he said.
Apple’s advantages over Meta can be seen in its expertise in combining services, software and hardware, including designing its own ultra-efficient chips. It also has an army of millions of iPhone app developers ready to rewrite their software to run on what is said to be called Apple’s “RealityOS” platform.
Sahar Fikouhi, founder of ARki, an iPhone AR app for viewing 3D architectural models, experimented with Microsoft Hololens but said only Apple’s platform could reach a wider audience.
“We are confident that Apple will be the company that produces a headset with mass market appeal,” she said.