VR vs. AR vs. MR: What Is Each One Good for?

Hardly a day goes by without the promise of some new, world-changing, product in the alternative reality space (currently a mash-up of VR, AR, MR, and now sometimes called XR). It can be hard to keep the technologies straight, so we’ve put together a short guide to the major aspects of each of them, and their relative strengths and weaknesses.

Virtual Reality (VR)

Due to the large amount of processing power required, initial VR headsets were all tethered — often with multiple cables for video and tracking. HTC broke through that requirement with a wireless add-on adapter, but now models like Quest from Oculus have combined inside-out tracking with on-board processing to provide a completely un-tethered solution. Its low-power processor can’t provide the same gaming performance as its larger sibling the Rift attached to a VR-ready Windows computer, but it’s certainly a lot easier to set up and use.

Augmented Reality (AR)

Annotating or projecting objects as overlays on the live view from a smartphone or tablet camera has proven very useful for applications such as industrial maintenance. As the cost of the needed sensors has come down and they’ve been included in consumer phones, consumer applications like visualizing furniture layout and AR-centric games have become increasingly popular. Without a doubt Pokemon Go quickly became the most-popular AR application ever, while its success still hasn’t been matched by anything since, despite the large amount of hype around AR.

Apple and Google have both officially gotten on board the AR bandwagon, with Apple’s ARKit and Google’s ARCore providing the raw material for AR app developers. There are some impressive apps for each environment, but aside from special-purpose applications like the Loew’s Furniture layout app, and a few games like Pokemon Go, applications based on them still haven’t become mainstream.

Mixed Reality (MR)

It didn’t take too long before the idea of having the best of both AR and VR in a single device started gaining ground, and Mixed Reality (MR) was born. The most famous MR device is certainly Microsoft’s Hololens. There are multiple ways to create MR devices. The most common is to use a natively-AR visor/goggle like Hololens does, but with sufficient projected display intensity to completely obscure portions of the actual scene, mimicking VR for a portion of the visual field. But native VR devices can also be used, by adding one or more cameras to the front, so that a “real world” image can be projected by the VR displays with additional virtual elements added as needed.

Unfortunately, Microsoft itself has seriously-muddied the waters by introducing what it calls “Windows Mixed Reality” devices. There really isn’t anything “mixed” about them. They are VR headsets that feature cameras to help with inside-out tracking — meaning they don’t require external beacons — but they do not allow mixing virtual and augmented experiences the way true MR devices do. Their use of the term MR really doesn’t make any sense, except to help distance their new generation of devices from the already-shipping “Windows VR” devices like the Oculus Rift and HTC Vive.

So, What’s XR?

One device I might put in the XR category is Google Glass. Glass is frequently described as an AR device, but unlike true AR devices, the Glass display is small and deliberately out of the way. It isn’t designed to overlay anything onto your view of the real world.

VR: The Best Way to Travel to Other Worlds

VR’s biggest strength is immersion. With a proper setup including full motion tracking, high-quality video, and spatial audio, you can start to believe in the reality of entirely artificial constructs. There is no more immediate demonstration of this than the beam-balancing demo. It takes some real mental effort to step onto the virtual balance beam, even though you know you are in a room with a perfectly flat floor. And panicking if you start to step off is a common reaction.

For creativity and gaming, the addition of touch controllers also provides a whole new dimension. Wielding a paintbrush or swinging a sword becomes a natural action. Well-implemented applications let you stay immersed using only the touch controllers, because needing to pull up your headset to find and use your keyboard mid-game is really annoying.

Birdly is by far the most immersive interactive digital experience I’ve ever gotten to try. You can just about convince yourself you are a bird, as you flap and turn through the sky above a city. It is not for the easily motion sick, though, or for those with a fear of heights. If you get a chance when visiting San Jose, it is absolutely worth the additional few dollars The Tech Museum charges for the opportunity. Like the other VR experiences in the museum’s exhibit, it is only for those aged 13 and older.

AR: The Best Way to Bring Things to You

Whether you want to populate your yard with pink flamingoes, or your living room with potential furniture purchases, AR is the preferred solution. It is also the best way to annotate real world scenes with additional data. That can be anything from additional information about a painting in a museum to the service instructions for an elevator needing repair.

Current AR solutions struggle to place objects realistically in the 3D real world, with heavily-funded Magic Leap being one of the leaders in using more-natural lightfield-based projection displays to try and solve the shortcoming.

Magic Leap has a well-deserved reputation for hyping its tech to the max, and this marketing text for its developer headset is no exception.

MR: Pushing Past AR and VR

Current MR devices are best thought of as either VR devices that add cameras so that you can baseline your virtual world with the actual world around you or AR devices that have sufficient display intensity to create a believable virtual reality in some portion of your visual field. No device currently exists that can do an excellent job of both VR and AR, so they all involve tradeoffs that favor one or the other.

While Hololens is certainly the most well known MR device, its limited Field of View means that virtual experiences are not nearly as immersive as when using a VR headset. Likewise, putting a pair of cameras in front of a VR headset and showing you the world in front of you on the headset’s displays is no substitute for looking out at the real thing. One of the most intriguing MR devices I’ve used was from Meta (now reborn as Meta View). It allowed you to create virtual desktops and TV screens at will, and by the final iteration was actually easy enough to use (initially grasping and moving was difficult) that I could imagine myself eventually using something like it.

Future Of XR

AR, VR, and MR technology are all advancing quickly, with new generations of devices rolling out each year. However, the “Holy Grail” continues to be finding applications that will drive them to mass market acceptance. By definition, VR is awkward and isolating, so the experiences need to be both compelling and more than just a novelty. And AR has to get accurate and powerful enough that it becomes the best way for people to accomplish everyday tasks. In the meantime, both industries are flourishing in a variety of commercial and niche applications, so expect continued progress on all fronts.