Apple, seeking a breakthrough product to succeed the iPhone, aims to have technology ready for an augmented reality headset by 2019 and could ship out a product as early as 2020.
Unlike the current generation of virtual reality headsets that use a smartphone as the engine and screen, Apple’s device will have its own display and run on a new chip and operating system, people familiar with the development said. The development timeline is very aggressive and could still change, said the people, who requested anonymity on the matter.
While virtual reality immerses the user in digital world, augmented reality overlays images and data on the real one. The applications for AR are endless, from a basketball fan getting stats while watching a game to a mechanic streaming instructions on how to fix a specific piece of equipment. Apple isn’t the only company working on the technology. Google, which drew derision from $1500 smartglasses a few years ago, is developing a business-oriented variant. Startup Meta has developed a headset with focus on education and medical uses.
Apple Chief Executive Office Tim Cook looks at AR as less isolating than VR and as revolutionary an idea as a smartphone. He has talked up the technology on Good Morning America and gives it as almost much attention during earnings calls as sales growth. “We’re already seeing changes that will transform the way you work, play, connect and learn”, he said in the most recent call. “Put simply, we believe AR is going to change the way we look at technology forever.” Apple declined to comment.
The company began putting together a team to work on AR-related projects a couple of years ago, Bloomberg reported in March. Led by Mike Rockwell, who previously ran engineering at Dolby Labs, the group has now grown to several hundred engineers from across Apple, the people said. Scattered across offices in both Cupertino and Sunnydale, California, the team is working on several hardware and software projects under the umbrella code name of “T288”.
The first product by the group was ARKit, software that outside developers use to create application for the iPhones and iPads, leveraging their screen, cameras and processors to create virtual 3-D interfaces for online shopping, education and gaming. This was an interim step, giving Apple an opportunity to test the technology on an existing product.
The next step – building a headset responsible for creating 3-D videos without draining the battery – is a much larger challenge. Cook acknowledged as much in a recent interview with The Independent, where he said, “Anything you see on the market very soon would not be anything we would be completely satisfied with.” Referring to challenges creating screen displays, Chief Design Officer Jony Ive told a tech pane last month that “there are certain ideas that we have and we are waiting for the technology to catch up with the idea”.
As with previous products, Apple isn’t waiting around to for someone else to create a chip powerful enough to control its AR headset. Its designing one in-house that is similar to design-on-a-package component in the Apple Watch. Such chips can squeeze more components – graphics chips, AI, CPU – into a smaller area than standard processors; they also consume less power.
The new operating system, internally dubbed ‘rOS’ for ‘reality operating system’, is based on iOS, the iPhone’s operating system. Just as tvOS powers Apple’s smart TVs, watchOS the Apple Watch series, ‘rOS’ will power Apple’s AR headset. Geoff Stahl, formerly a software developer for games and graphics at Apple, is one of the directors of the ‘rOS’ software group.
Apple hasn’t finalised how users will control the headset and launch apps, but is investigating touch panels, voice activation via Siri and head gestures. Engineers are prototyping a range of applications, from mapping and texting to more advanced features including virtual meeting rooms and 360-degree video playback. The company has decided pairing the headset with its own version of the App Store, where users will be able to download content, just like the Watch, iPhone, Apple TV and Mac.
Because Apple doesn’t have a fully functioning headset of its own, engineers have begun using HTC Vive headsets for testing purposes. They’re also working on a device that works like an Oculus VR headset designed for an iPhone’s screen, cameras and chipset. Apple doesn’t plan to sell the gadget but instead intends to use it internally to test AR apps next year.
With the headset at least two years away, Apple wants to make it easier for AR developers to bring new features to the iPhone. To that end, the company plans to add a new version of the ARKit software tools as soon as 2018. They could help create software that remember accurately where a digital object was kept in a virtual space, a technology known as persistence tracking; you look away ad the object is still there. The tools will also make it easier to create AR games for multiple players.