Understanding mixed reality headsets: Will they replace smartphones?

What is mixed reality?

The term “mixed reality” was first coined by Paul Milgram and Fumio Kishino in their 1994 paper, “A Taxonomy of Mixed Reality Visual Displays.” They explored the idea of a virtuality continuum and how different types of visual displays can blend the real and the virtual. And that’s how the technology was born. Modern mixed reality headsets combine the best of both virtual reality and augmented reality, allowing you to experience and interact with both real and digital worlds in a natural and immersive way.

Imagine wearing a headset that can scan your surroundings and project realistic 3D holograms onto them. You could play games with virtual characters, learn new skills with interactive tutorials, or collaborate with others on creative projects. But it’s not just about the visuals – there’s also a focus on the auditory experience. You can hear spatial sound that matches the direction and distance of the objects you see with mixed reality headsets, offering unmatched immersion.

All this has become a possibility thanks to the strides made in computer vision, graphical processing, display technologies, input systems, and cloud computing over the past couple of decades.

How does mixed reality work?

In the most basic sense, an MR headset works by using advanced AI sensors, cameras, CPUs and GPUs to process and store data in three dimensions. But to truly understand how mixed reality works, we would first need to understand the mixed reality spectrum.

The mixed reality spectrum is a way of describing how physical and digital worlds can be combined in different ways to create immersive experiences. On one end of the spectrum, there is physical reality, where we interact with real objects and environments. On the other end, there is digital reality, where we are fully immersed in a virtual world. In between, there are various degrees of mixing physical and digital elements, such as overlaying holograms on the real world, or blending real and virtual environments.

An excellent real-world example of how these headsets blend realities is Vision Pro’s seamless integration of apps with the environment. Users can effortlessly browse through various options by flicking their hands in the air and selecting their desired app by tapping their fingers together – all while still being in touch with their surroundings.

Virtual reality vs augmented reality vs mixed reality: What’s the difference?

Now that we understand that mixed reality is a spectrum, it’d be easier to differentiate MR from virtual reality and augmented reality.

Virtual reality (VR) is tied the closest to the virtual world and immerses users in a fully artificial digital environment, where they can only see and interact with computer-generated images. It’s also the most popular kind of headset. Augmented reality (AR) keeps users more in touch with their surroundings, overlaying virtual objects on the real-world environment and enhancing the user’s perception of reality. Mixed reality (MR) not just overlays but also anchors virtual objects to the real world, allowing them to interact with each other and the user.

How are mixed reality headsets being used today?

Education: Mixed reality headsets can enhance learning by providing realistic simulations, interactive content, and personalised feedback. For example, Microsoft HoloLens has been used to teach anatomy, chemistry, and engineering in various universities.

Entertainment: Mixed reality headsets can offer immersive and engaging entertainment options, such as games, movies, and social media. For example, Magic Leap One has been used to create a mixed reality version of Star Wars, where users can interact with characters and environments from the franchise.

Healthcare: Mixed reality headsets can improve healthcare by providing visualisation, diagnosis, and treatment support.

The Trimble XR10 with HoloLens 2. (Image: Microsoft)

Industry: Mixed reality headsets can enhance industry by providing training, collaboration, and productivity tools. For example, Trimble XR10 has been used to create a mixed reality solution for construction, where workers can access 3D models, instructions, and data on-site.

The future of mixed reality headsets

Smartphones have a major drawback: they distract us from the real world and make us focus on a tiny screen, cutting us off from reality. Mixed reality headsets can solve this problem. They create a more immersive and interactive experience than smartphones, by merging the physical and digital worlds. Some experts even foresee that mixed reality headsets will be the next computing platform, after the evolution from desktops to laptops to smartphones. For example, Apple is rumoured to be working on mixed reality headsets that will eventually replace the iPhone.

However, mixed reality headsets are not without flaws. They are still costly, heavy, and uncomfortable to wear for long durations. They also face technical hurdles, such as battery life and latency. Furthermore, mixed reality headsets in their current form may not be socially acceptable due to their large, conspicuous appearance which can isolate people.

Frankly, it is difficult to predict when mixed reality headsets will replace the most common gadget in the world today, but if they overcome these challenges, they will undoubtedly surpass smartphones. The question is not if mixed reality headsets will replace smartphones, but when.