Creatives and platform developers have been actively investing in and moving forward with Virtual Reality (VR), Augmented Reality (AR), Mixed Reality (MR), and 360 Video technologies for some time now.

While there are differences between all of these, they all still bring an immersive experience to the viewer, and they mostly - depending on the use-case - require audio.

It is true that the domestic customer base for immersive media is still relatively low, though a good proportion of the population has experienced it in some way, and an increasingly wide range of applications and markets are starting to embrace its potential.

So, as the stock of content that provides a fully immersive audio and video experience increases, and the cost of that experience decreases, the production and creative norms of that content will become more and more familiar to the general population, and will drive expectation in all mediums, including broadcast. Content creators and their tools need to evolve or be re-imagined.

Which Reality Do You Require?

VR, or Virtual Reality, normally refers to an entirely artificial environment. For example, putting on a VR video game might put you inside a car with a view of the race track while you’re sitting on your sofa. Or a VR tour of an architectural concept can put you inside a building that hasn’t been built yet. Virtual reality is mostly experienced through dedicated headsets such as the Facebook Oculus products, the HTC Vive, or the Playstation VR headset.

Cinematic VR probably has the most content available, though much of it is short-form, due to the cost of production and the current speculative commercial nature of releasing a VR experience.

Documentaries are a fantastic VR medium, and one where its unique ability to short-cut to empathy can be used to great effect. VR can provide an ultra-real 1st-person experience that does all the heavy lifting when you want the viewer to put themselves into the shoes of a protagonist.

VR games are impressive, but they are still relatively rare. The installed user-base of VR headsets has not grown as rapidly as many had first hoped, though maybe a new generation of lower-cost, standalone headsets such as Oculus Go and Quest will push the uptake on. The units put not just VR content but also ‘normal’ video and music streaming services on a much more convenient and affordable platform.

AR and MR

AR (Augmented Reality) and MR (Mixed Reality) do have separate definitions, but more recently the terms have been mixed up, with AR remaining the one most people know.

AR builds on what you can see around you by bringing in imagined or created elements that are not actually there but are superimposed on the real world. MR refers to interaction between reality and its augmented aspects. That is, if you put a virtual coffee cup down on a real table, you would want to hear it touch, from the correct point in space, and not see it go through the table.

AR and MR can be experienced through a ‘magic window’-type app on a smart device or gaming console, or through dedicated headsets such as the Microsoft HoloLens or the Magic Leap One.

AR Made Real

Experiments in AR have been going on since the 90s, though early viable gaming implementations came along with products like the AR implementation in Nintendo’s 3DS, or the cute Sony Eye-Pet for PlayStation 3. Pokemon Go and the forthcoming Harry Potter: Wizards Unite are more up-to-date examples that use the magic-window concept.

Some high-profile AR landmarks include events such as the opening ceremony of the 2018 League of Legends world championship. That event boasted more viewers than SuperBowl (it reportedly peaked at around 200 million combined concurrent viewers on outlets such as PandaTV, Huya, YouTube, Zhanqi and Egame), and featured the debut of the Augmented Reality K-Pop group K/DA, which combined imagined champions from the game joining with real-world artists live on stage for a performance of POP/STARS.

At the 2018 SXSW event the Sennheiser AMBEO team demonstrated an augmented live music performance where a live vocal performance by Noa (Playing Savage), was augmented by a full virtual backing band with immersive audio, via an iPad and the AMBEO Smart Headset. This demo evolved into the AMBEO Augmented Audio Lab developed for the Magic Leap glasses, and enabling participants to create a spatial soundscape in which they can wander. 

More utilitarian uses of AR and MR might include being able to look around your kitchen and see or hear the status of your appliances and storage areas, or a surgeon might be able to have reference materials in their field-of-view, along with remote, live consultations. Developments such as the new Microsoft Dynamics 365 Guides for HoloLens looks like it has some serious potential in areas such as manufacturing and engineering.

Another very public use of immersive media is in the exhibition market. New media is a common subject matter for targeted pop-up exhibitions at museums, and the technology itself is a very attractive way to draw in the crowds. A recent Pink Floyd experience at the Victoria and Albert Museum in London got nearly 400,000 visitors in only a few months with an AMBEO 3D presentation of Pink Floyd: Their Mortal Remains, which put visitors in the audience for the live recording.

The Content Stockpile

The consumer needs a critical mass of content to be interested in investing in something - in gear, in the content, and in a medium. That’s why creators need to be a focus point for the manufacturers and technology leaders: We need cool content and we need amazing apps that will then feed demand from consumers.

Magic Leap just closed off its first round of funding in its Independent Creator Program, and another round is promised for 2019. Sennheiser has its own AMBEO Developers Program, which includes free access to the AMBEO Augmented Audio SDK for iOS. You can apply here: http://sennheiser-ambeo.com/developers-program/

Some great leaps have been made in developer tools, such as the fantastic dearVR Spatial Connect workflow and plug-ins from Dear Reality, or the GUI leaps that many of the well-known console and DAW manufacturers are currently developing and launching. Include the rise of 360 video, ambisonic support on YouTube, and clever promotions such as Google Cardboard, the environment looks ripe for an immersive revolution.

When the content dam breaks, the consumer will be immersed - in all sorts of ways. That will increase awareness of the values and benefits of immersive audio, which will in turn become a benchmark. Broadcasters can only benefit from meeting that mark.