More articles in this series and about OTT/Streaming:

The phrase “immersive viewing” inspires visions of recent films like Ready Player One, Avatar and Travelling While Black. Yet the concept of immersive experiences is centuries old, it goes at least as far back as anamorphic art in the 15^th century Renaissance Period. But in today’s tech-fuelled world, while the film representations are still science-fiction, the reality is getting a lot closer. What does this mean for how we need to create content for immersive experiences, and how we need to prepare to deliver video in the future?

Why Immersive Experience Matters

Watching TV, listening to radio, or being at a live theatrical or sporting event is a passive experience. We sit back and watch and listen. Of course, we might get really excited about what we are experiencing, and we can feel very engaged, but we are not participating. Immersive experience is about participation. Immersive experience is the ultimate way of experiencing a story without being the subject of the story.

Consider for a moment an athlete or an actor who is performing. They are the subject. They are directly experiencing what it is like to be that person, and they are directly engaged in telling and experiencing the story. But what about the spectators? Unless they participate, they are not involved in the story itself, but are simply onlookers, observers. Immersive means that spectators can fully experience the story. They can sense the whole environment of the story – the cheering crowds, the wind and sun, the other actors in the story, and more.

Immersive is important because it is the most sense-inspiring way to experience a story. It creates new possibilities for interacting with our environment. From a commercial perspective, it is the ultimate marketing method to plug into, which is why Meta, Google, Apple, and others are fully behind the trend. It is also why sports rightsholders, program producers, entertainment companies, and even medical and education businesses, are backing this approach.

True immersivity was lost in recent times because we lost the notion of participating in what we were watching. We have become used to the passive screen experience. It is the power of technology that is reversing this situation and giving us the ability to participate. The catalyst of the revolution is the pervasiveness of the technology.

Immersive experience over the last 5 years has been considered by brands as an extra way to conduct marketing and sales. But alongside the evolution of technology comes the evolution of how we implement it to create new experiences, and so today immersive can be seen as central to the experience of a brand, which has wide-reaching impacts across all economies and societies.

The Ethics Of Choice

Immersive is a way to look at an environment with spatial awareness of that environment. The engagement is therefore different in terms of how you interact, and how a content creator builds an ecology around their content.

Immersive carries with it an ethical dilemma – can immersive cause people to disconnect from the real world and become addicted to the non-real world created by the technology? Adipat Virdi, immersive experience maestro and consultant, and recently the Global Creative Product Lead for Immersive at Meta, states, “The key point is that immersive experience is about choice. We have choices in all areas of our lives, some of which can be dangerous or lead to an addiction. Immersive experience is a choice versus a 2D-screen experience. It’s about choosing how close a person wants to get to the content, how much they want to engage with the characters, and be part of the community. As with most technology advances, we learn and adapt through cycles of evolution. Immersive is no different.”

Immersive experience is also about gamification, but it is not about gaming. Choice is a core building block of gamification. More people are engaging with gamification, at a personal level and in the corporate world. As content is created, and as the level of immersion is considered by the content creators, there is a choice to make about how much gamification should be injected into the content.

Adipat adds, “There is an important distinction to note in the area of content creation, which is that the screen should not be the defining factor in the way content is created. More engagement equals more immersion which equals more choice. The question for content creators to consider is therefore how they build an ecology that offers this spectrum of choice to the audience. The overarching point is that the defining factor in the way content is created should be the type of engagement we want to create.”

What Is Driving Us Forwards?

This move to immersive experience is driven by a human need for connection and control. In general, thanks to the internet and telecommunications advances, the world is now a much smaller place in terms of how we can connect with each other. Younger generations have developed a greater sense of self-identity and agency (agency is defined as “the sense that I am the one who is generating the experience represented on a mental map”, versus ownership which is defined as “the sense that I am the one who is undergoing an experience, represented on a mental map”; source: USA National Library of Medicine). Technology advances, created and driven by new generations to fulfil new consumer demands, are supporting this drive forwards to more immersive experiences.

The technology of immersive experiences starts with content capture which includes a 360-camera rig and a microphone tracked to North for spatial awareness. Next, for content production, the technology involves spatial computing driven by Web3. Then, for content delivery, we will rely on the same edge compute and content delivery infrastructure as video streaming, plus a device that can display the environment ranging from a Smartphone to a full VR headset. In essence, the viewer experience changes from navigating through 2D web pages to navigating through 3D web spaces.

The recent launch of the Apple VisionPro is a sign of growing focus in this area. We are still in the early days of immersive experience technology and while in this early-adopter stage the technology leaders like Apple and Meta are focusing on primary use cases where the value is greatest, like entertainment, healthcare, construction, and retail. Over time, as adoption matures, not only should these use cases scale up but other use cases like manufacturing and education will grow up from their own early stage of immersive evolution.

Future Expectations

Looking to the future, Apple have already described VisionPro as a computer you look through, not a computer you look at. It is a clever way to move from early adopters to mainstream adoption. It captures the imagination of where the technology is going. Over time it is expected that the consumer hardware will get smaller, until we’re in the world of contact lenses rather than head-sized goggles.

A rough estimate is that we are 10 years away from a time where immersive experiences are delivered in this more comfortable format. To reach this point, consumption needs to become pervasive and drive the technology developments. We must be able to scale spatial content and immersive experiences. Younger generations are the driving force for this, already moving beyond the simple 2D screen experience into more immersive experiences by engaging with communities and using multiple screens simultaneously.

But it’s not just the consumer equipment we need to think about. We need serious network capacity for high-scale delivery use cases. A VR headset draws in data at about 200 Mbps. If 10,000 people are interacting together in a single location like a stadium, then that single area needs to support 2 Tbps of concurrent viewing. Wired up correctly, it’s doable. But at scale in the home, there will be challenges to overcome. As noted in our article discussing A National Blueprint For Video Streaming Delivery, normal streaming video at a good 4K or UHD resolution will require about 10Mbps per device served. 200Mbps is 20 times that figure. And yet for large-scale streaming audiences at 10Mbps we already require streaming video infrastructure capacity about 10-20 times larger than we have today, based on average bitrates and peak audience sizes on streaming platforms. If immersive experiences start to dominate media consumption during prime-time viewing in the evenings, we could see that capacity requirement jump up to 100 times more than we have today.

Immersive experience is closer to multi-player gaming than video streaming. This means that interactivity is an important feature of the experience. We need to technically scale our processing and delivery infrastructure to handle this type of heavy edge computing workload. The key requirement for interactive use cases such as gaming together or watching a live event in a virtual stadium together will be synchronicity between users, which will require management of latency. If a user has much lower latency than their fellow-user, then latency will need to be controlled to the level of the slowest latency. To achieve the lowest and most stable latency will require the servers to which each user is connected to be synchronized with each other. A CDN can manage this synchronization between edge servers, thereby guaranteeing the users’ QoE (Quality of Experience). But because bandwidth requirements are expected to be high, the pressure on the CDN and ISP networks will also be high. We need to scale the infrastructure above and beyond normal streaming video to handle this.

But how fast can we expect the demand to grow? Perhaps more quickly than we might expect. As Evan Shapiro, Media Universe Cartographer states: “There are 3.6 billion gamers on earth. Nearly 60% of all kids 8-12 play Minecraft. The majority of kids under 15 game every day. More than half of gamers 8-17 say they use online games to “hang out with friends.” The next iteration of social media is growing on gaming platforms, right now. What’s more, these aren’t just entertainment platforms, they are fully functioning economies. As Gen Z and Gen A mature, their gaming habits will be adopted into mainstream non-gaming platforms and experiences. If you want to know where media is headed, spend some time with your kids on Fortnite and Roblox.”

What Is The Next Step For Immersive Experience?

Consumer demand drives us to develop entire societal ecosystems. It makes producers of content, products, and services justify the value of what is being produced. Large and very profitable businesses, like Apple, Meta, and Google, can naturally take a lead in investing in innovation for new content and new consumer experiences.

Today, immersive experiences are arguably loss-making. But there is latent demand, particularly from younger generations who are already demanding experiences that are ahead of what is available in mainstream media ecosystems. Mainstream media should be market testing minimum viable products, to ensure they are providing the younger generations with the type of content they are expecting in the years ahead, and to identify how ecosystem-wide value can be created.

“Telling a story for an immersive experience, involving choice and gamification, is a skill that must be developed in the media industry,” states Adipat. “The boundaries of how one must do something in media are breaking down as the need for an ecology of experiences displaces a hierarchy of experiences. Content producers and providers are often worried about cannibalizing existing business with new immersive content experiences, and therefore the media industry is a late adopter of immersive. But the problem is that TV audiences are early adopters. This requires solving in order for the traditional media industry to retain next-gen audiences.”

In conjunction with creative storytelling skills, today’s technology allows us to create and deliver content in compelling immersive formats. Scaling infrastructure for the wide-scale delivery of immersive experiences will require expansion of video delivery capacity, but this is already underway and incrementally happening as 2D video streaming delivery grows. Fiber broadband speeds are already sufficient to support the bitrates required and need to be scaled out to all households. But there is work to do to distribute content delivery infrastructure nearer to consumers to reach high efficiency at scale, or else the extreme demand on networks created by immersive will cause significant viewer experience issues.

Consumer demand once again pulls streaming video inexorably forwards. And just like the onward growth of video streaming, Immersive Experiences will require us to continue building out the right technology to support it efficiently and sustainably.