Apple VR headset and augmented reality smart glasses: The story so far
Apple boss Tim Cook is a big fan of augmented reality (AR). In fact, he has publicly praised the technology since at least the summer of 2016, when he said on an earnings call how Apple sees great opportunity for the tech.
The company initially chose not to engage with AR's sibling, virtual reality (VR). So while Samsung, Google, Facebook (via its Oculus division) and others all fought to take VR mainstream, Apple saw those clunky headsets, occasionally made of cardboard, and sat it out.
Read More:
In the meantime, as those companies created waves at first but later struggled to keep developers and consumers interested, Apple — and Cook personally — continued to beat the AR drum.
The message was, this is the future technology worth fighting for. The runaway success of Pokémon Go, a mobile game with AR components, only helped bolster that position. At around the same time, in 2016, Apple gave its developers ARKit, a software development platform to help them produce augmented reality applications for the iPhone and iPad.
Cook's AR evangelism continues
With Cook's passion for AR continue to bubble away, it was in late-2017 that he began to talk even more bullishly about the technology. Speaking in October that year, Cook suggested that the adoption of AR, although slow at first, will eventually be as dramatic as that of the iOS AppStore and even of multi-touch, both regarded as foundations of the iPhone.
"Thinking back to 2008, when the App Store went live, there was the initial round of apps and people looked at them and said, 'This is not anything, mobile apps are not going to take off,' he said in 2017. "And then step by step things start to move."
Cook added: "And it is sort of a curve, it was just exponential — and now you couldn't imagine your life without apps. Your health is on an app, your financials, your shopping, your news, your entertainment — it's everything. AR is like that. It will be that dramatic."
Then, in the second half of 2019, things really began to accelerate for Cook and Apple's love affair with AR. First came a report that Apple had cancelled its augmented reality smart glasses, but this was quickly debunked two months later when code referring to augmented reality was spotted in iOS 13, Apple's latest iPhone and iPad software.
Apple's AR project is alive and well
Discovered in an internal build of iOS 13 not intended to leave Apple HQ, references to an app called STARTester were discovered. What was interesting here is how the app can, according to the leaked internal iOS 13 build, be switched in and out of head-mounted mode, suggesting it could run on a headset of some kind, as well as an iPhone.
Further evidence was found in a README file, also within the build of iOS 13, which described a 'StarBoard' system shell for augmented reality-enabled apps. A device codenamed Garta was also mentioned by the leaked software. Further references to augmented reality (AR) included various 'views' and 'scenes', with references like 'ARStarBoardViewController' and 'ARStarBardSceneManager.'
At this point, it was speculated by analysts that Apple was on track to launch its first AR hardware a year later, in 2020, and that device would resemble an improved version of the Google Glass smart glasses.
Then, also in September 2019, it was reported that Apple had acquired Colorado-based Akonia Holographics, a six-year-old startup that makes glass lenses designed for augmented reality devices.
What iOS 13 revealed
Additional clues for a future Apple AR device arrived in the public version of iOS 13 for iPhone. Steve Troughton-Smith, an iOS developer, discovered further references to StarBoard. "Guess secrecy is out?" the developer tweeted.
Troughton-Smith continued on Twitter: "The GameController framework in iOS 13 also has a gamepad profile for a device meant to be used while using stereo AR apps. The controller profile has a clicky trackpad, a trigger button, and a system (home?) button. Handheld controller for Apple's headset?"
That description sounds similar to the hand controllers used alongside other AR and virtual reality headsets, like the Oculus Rift, PlayStation VR, and Microsoft HoloLens. Further digging into iOS 13 suggested that the AR headset "...has a dashboard of some kind that shows you your available Stereo AR apps (like CarPlay)," Troughton-Smith said.
Patents give away more details
Next up, a patent filed by Apple on October 24, 2019 revealed how the company is looking at a way to fit transition lenses to a wearable product, describing a "head-mounted device with an adjustable opacity system." This is similar to Google Glass, where a user interface is projected onto a transparent display on a head-worn device, right in front of the user's eye. But, unlike Glass and more recent smart glasses like the discontinued North Focals, Apple's plans are for the opacity of this display to be adjustable.
This would see the use of a photochromic layer on the lens, adjusting how much light passes through and into the wearer's eyes. Like reading glasses with transition lenses, this could help protect the user from bright sunlight, then go clear again once in the shade.
In November 2019, a report from The Information claimed Apple hosted a presentation a month earlier that filled the 1,000-seat Steve Jobs Theater at Apple HQ, briefing staff on its augmented reality plans.
These plans reportedly included the launch of an AR headset in 2022, followed by sleeker AR glasses a year later, in 2023. This contradicts the expectations of the often-accurate Apple analyst Ming-Chi Kuo, who had repeatedly said Apple's first AR device would arrive in 2020. As 2019 drew to a close, the 2022-2023 timeline felt more realistic.
That report also claimed the headsets will have a comfortable fit, and Apple would start speaking to developers about building software for the AR headset in 2021.
Virtual Reality to be Apple's first step
Despite earlier indications that Apple was happy to sit out the VR race and instead focus on AR, it now – as of early 2021 – looks like the company will head down the VR route after all. Reports by The Information and Bloomberg have both suggested as much, with a mixed reality VR/AR headset tipped to arrive in 2022. This is claimed to resemble other VR headsets, but with more impressive specifications and an enormous price. It is claimed the headset will have a pair of 8K displays, over a dozen cameras for tracking hand movement, Lidar for mapping the user's environment, and a price tag of $3,000 – or ten times more than today's best-selling Oculus Quest 2 VR headset.
It is claimed the headband will be similar to the rubber sport strap of an Apple Watch, along with the same mesh fabric as used by the HomePod, and there will be eye-tracking technology too. This will reportedly be used to understand where you are looking, then lower the resolution of areas in the wearer's peripheral vision to help optimize performance. It is claimed the headset will run on chips related to Apple's own M1 silicon used by the 2020 MacBook Air and Pro. This would mean the headset is untethered and not powered by an iPhone, Mac or other control unit.
Apple is said to be working on a number of ways to control the headset, including hand- and eye-tracking, as well as a dial on the side of the headset (potentially like the Digital Crown of the Apple Watch and AirPods Max), and what is described as a "thimble-like" accessory, although nothing has been finalized just yet. The design is said to resemble a "sleek, curved visor attached to the face by a mesh material and swappable headbands".
At this point it isn't clear if Apple intends to sell this to consumers or to the enterprise market. We can see such a headset being useful for designers and engineers, but wonder if enough consumers would be willing to part with $3,000 for such a product. We must also apply the usual caveat and say technology companies like Apple often work on future devices that never actually launch, and at this stage the headset described here could still fall into that category. Remember the rumors of Apple making its own television?
What about the AR glasses?
Claimed to be called Apple Glass, the company's AR spectacles are still reportedly in development (as of early 2021), and are said to resemble normal glasses and could be priced at around $500. However, the device is reportedly still in the early stages of development, is at least several years away from a commercial launch, and may still be cancelled. The smart glasses are said to use OLED micro displays from Sony and have a 0.5-inch display with a resolution of 1280 x 960.
Instead of a standalone device like the VR headset, renowned Apple analyst Ming-Chi Kuo believes the glasses will act as an iPhone accessory, like the Apple Watch does today.
It is safe to say Apple is actively working on AR and VR products, but that these are still a long way from reality. They could both be cancelled in the coming month or even years, and if they do eventually launch they could be quite different to what has been reported so far. All of that said, we look forward to seeing how Tim Cook plans to realize his AR ambitions.
Vuzix Blade AR Smart Glasses, with Amazon Alexa Built-in, HD Camera and Voice-Controls
