Currently it is dominated by the “big three” of 360-degree video, Virtual Reality and Augmented Reality, but object based surround sound is also emerging to make a significant contribution, while a few CE vendors such as Philips are promoting some form of ambient lighting as another attractive feature. 

But being a collection of technologies, it may not be the providers of each component that can offer the most compelling and complete immersive package. At recent trade events including TV Connect held in London during March 2017, French TV and digital home software vendor SoftAtHome has been presenting its vision of the immersive future. This features its own set up called Video 360 on TV, combining 4K, HDR (High Dynamic Range), Dolby Atmos audio and converged services around a TV portal. This allows users to navigate via a remote control within the TV image, enriched with the light ambiance and object based audio which exploits five speakers positioned around the room to isolate sounds associated with particular events such as the passing of a car across the screen in real time. It matches the sound balance with the event’s progression from frame to frame.

SoftAtHome naturally promotes the value of its own SOP8 software as an ideal platform for integrating the ever-expanding number of components associated with immersive TV, which will include HFR (High Frame Rate) and elements of both VR and AR, as well as HDR, 4K and object audio. It highlights the potential of immersive TV to bring to life such software platforms which until now have been rather kept in the background struggling for attention, with their operator customers reluctant to credit them too much for the features of their services. 

It has allowed SoftAtHome’s Director for Innovation, Product Marketing and Pre-sales David Souhami to wax lyrical about the future of immersive TV as it embraces other human senses beyond sight and hearing to enrich the experience further. He outlined the company’s strategic vision to make TV ever more stimulating by engaging all the senses, including touch, smell and perhaps even taste indirectly. As he pointed out this may not just improve the experience for existing TV viewers but also extend it to people who for various reasons, such as poor eyesight, cannot enjoy it to the full at the moment. 

Even when it comes to the visual display itself there is vast scope for improvement beyond just raising the resolution or even the dynamic range, Souhami insisted. “We want to provide real 3D viewing, with proper depth of field and adding information relevant to the viewer within the display,” said Souhami. “Of course we are bringing in 4K, HFR, HFR and WCG (Wide Color Gamut), but the world is much bigger than the TV is today and so want to create a much larger surface. In the longer term the TV will no longer have a fixed edge and the picture will be able to move from the screen onto the wall so that it will no longer be a fixed display.”

This will in turn drive demand for higher pixel resolutions, to support large displays and also multiple angles or zoom ins. “By having 8K or even 16K, the service will help the consumer become an actor and no longer just a passive viewer.”

To some extent audio is closer to the ultimate experience given that SoftAtHome can already demonstrate surround sound with five speakers, two beside the TV on each side, one in the center and two at the back. “With Dolby Atmos we will go to 7 speakers, adding two at the top, so that finally sound comes from everywhere,” said Souhami. “With object audio, each sound component will have position to help recreate the real audio experience.”

Immersive TV is also about “surround video”, putting the viewer at the center of the scene rather than looking in from outside as with conventional 2D viewing, even when boosted by the various components of Ultra HD. Souhami talks about being on the cusp of something new, which will harness the techniques of VR but dispense with the need for wearing a headset, which he sees as an impediment to immersion and can never be part of the mainstream viewing experience. That is surely right as can be seen from the failure of 3D TV to win over mainstream audiences, although that also had a lot to do with the inability then to understand or take account of how human vision works.

SoftAtHome may enjoy evangelizing the virtues of ambient lighting but may find it harder persuading consumers to pay for and install the necessary equipment. Souhami is adamant though that lighting has a major role to play in immersion. “When you are outside it may be either dark or very light,” he noted. “That ambient light can help recreate that experience indoors, allowing the color to move around you and fill in the background. “As everything goes dark on the TV, it should go dark around you as well. Or if there is an explosion on the TV, if there is a burst of light around you as well, you feel more like you are inside it.” 

Another dimension to immersion is interactivity and here the major players such as Apple, Amazon, Google and Microsoft are promoting voice recognition as a key ingredient with respectively their Siri, Alexa, Assistant and Cortana technologies. But Souhami sees this as just a first step towards true conversational assistants. “We have to create new devices that can listen, interact and answer questions,” said Souhami. It has to understand context, so that if for example I’ve just come in from work it should begin with “Hello” or “Welcome Home”. 

Souhami also touched on the other senses, including smell, but admitted that posed technical challenges that have yet to be met, including “olfactorial latency”, which might mean a given smell lingers long after the associated action on the screen has passed. Perhaps tactile experience will be easier to recreate, having already been promoted by the UK start-up Ultrahaptics, which has exploited ultrasound to simulate “tactile feedback”. This technology directs ultrasound waves to project sensations through the air and onto the user, pinpointing areas of the hand that could be stimulated to evoke different emotions. Users have reported experiencing the sensation of raindrops rolling across their palms. 

Researchers at the University of Sussex are exploring this and other tactile techniques, having found that short, sharp bursts of air administered during a program at the right time to the area around the thumb, index finger and middle part of the palm can generate or enhance excitement. They even claim that slow and moderate stimulation of the outer palm and the area around the little finger can create sad feelings. This of course is very much work in progress and may arouse skepticism, but is the sort of development SoftAtHome is following.

There is of course the question of how to monetize immersion, but SoftAtHome’s view seems to be that opportunities will emerge naturally as a result of the improved experience and deepening viewer engagement. Early evidence suggests that may be right, with one example being an experiment in virtual reality to help market holidays by travel group Thomas Cook, conducted around March 2016. It worked rather like a flight simulator in the cockpit of a plane featuring real pilots and cabin crew, inviting viewers to find clues to the destinations. Although this required wearing a VR headset, it proved successful in achieving a 190 percent boost in sales of excursions from the UK to New York. 

Other successful commercial examples include a 360-degree ad from Hong Kong airlines which proved 35 times more effective than the same traditional 2D ad. Then a recent report from BI Intelligence, The Immersive Video Report: How VR, AR, and 360-degree video are shaping the future of content creation, analyzed the benefits of each of the “big three” immersive components, VR, AR and 360-degree video, concluding that each one could help brands and publishers flourish. It was especially positive about 360-degree video, which it argued “presented an opportunity for advertisers to reach massive audiences, while allowing viewers to engage with ads as they see fit. The format also has the potential to generate longer viewing times.” AR, which blends the physical and digital worlds, had great longer term potential and is forecast to expand at nearly 81% compound annual growth rate (CAGR) from 2016 to 2024, according to Global Market Insights.

VR is the most complex of the three, but also has the greatest potential, according to the report. VR content was found to elicit higher emotional engagement and longer engagement periods than traditional 2D, according to both YuMe and Nielsen. BI Intelligence predicted that global VR headset shipments will increase 359% over the next six years, from 12 million in 2017 to just over 55 million in 2022.

A lot of that headset growth though will come from gaming. As SoftAtHome argues, in the longer term immersive TV will only achieve its full potential by dispensing with that need for headsets.