One solution is content-aware encoding processed with artificial intelligence (AI) algorithms and advanced distribution codecs like EVC, VVC and LCEVC, which are showing promise by smartly encoding 8K content into four 4K streams that are delivered and upscaled by 8K TVs. This saves bandwidth and reduces distribution costs.

At a virtual panel discussion during the recent NABShow Express online-only trade show, experts discussed the technical issues needed to launch an 8K streaming service for consumers. Perhaps tellingly, when the moderator Chris Chinnock, Executive Director of the 8K Association (a consortium of 24 companies actively promoting 8K products and services) asked them to predict when such an ultra-high-resolution service would be financially successful in North America, they all hesitated and then suggested 8K highlight clips to support coverage of the 2022 Olympics and World Cup would make the most sense for a business model.

The panel, entitled “Delivering 8K Content to Consumer Homes” included Bill Mandel, vice president of Industry Relations at Samsung Research America and Thierry Fautier, President-Chair at Ultra HD Forum and vice president of Video Strategy at Harmonic.

Chinnock began the discussion by rattling off current 8K services in Japan (NHK) and Europe (streaming providers Chili, Megogo and The Explorers). NHK is using a satellite-based approach delivered at 85 MB/s while others are streamed online at 25-30 MB/s. You can also watch 8K content on YouTube and Vimeo—mainly nature documentaries. Much of this content has been shot in 8K (or 4K) but not posted in 8K, due to storage and other hurdles.

Harmonic’s Thierry described an 8K production of a soccer match in Argentina and some tests at the 2019 French Open tennis tournament that he said “looked very good” after the fact but noted that delivering a live event is still problematic.

“I think the glass is half full right now,” Thierry said. “We have to figure out what is possible on the technical side. What are the devices and what are the services?”

Different Codecs For Different Folks

Mandel said that the Explorers app on Samsung 8K TVs is a good start because it gives viewers something to watch on compliant TV sets now available from LG, Samsung—which now offers the 85-inch Q950TS 8K TV—and Sony. He added that the Samsung S20 phones that came out a few months ago are capable of recording 8K SDR video.

“You can watch [8K content] by putting it up to YouTube and then bringing it back down to your 2020 Samsung TV,” Mandel said. “There’s quite a bit of content on YouTube, more than just the documentaries and stuff. If you do some searches the Korean Broadcast Service has a model for music [a K-Pop channel] on YouTube in 8K. All of that can be watched through the YouTube app on 8K smart TVs. So, the content is coming but it’s just starting.”

However, getting that data-hungry signal into people’s homes requires reducing file sizes as efficiently as possible. Codecs being discussed include the Alliance for Open Media’s AV1; Essential Video Coding (EVC); Low Complexity Enhancement Video Coding (LCEVC); High-Efficiency Video Encoding (HEVC); and Versatile Video Coding (VVC). All have shown promise in getting 8K files down to 4K data rates. This makes the content suitable for today’s cloud-based infrastructures, as the cloud provides much more compute capacity than was possible before.

“The benefit of HEVC is that it works on all of the devices that exist today being for 8K viewing, so there is a lot of benefit to using an existing codec on existing devices,” Thierry said. “I think for the next three years, we will be using HEVC and AV1 on supported devices. And when the demand ramps up, we can think of next-generation codecs like VVC or EVC.”

Content Aware Compression

A new type of compression algorithm, called “content-aware” encoding has begun to find a following among some of the major TV set manufacturers and content delivery services. Content providers call it “context” aware encoding (where the video will be consumed), but they all use machine learning and deep video analysis to achieve optimum quality for each video with the fewest bits necessary. This a concept where the encoder dynamically decides on the bit rate required based on the complexity of the content. For example, at the Harmonic/French Open trial last year, they took a live stream at 65 MB/s and encoded it down to an average bit rate of 27 MB/s, using AI to perform the best compression.

“Using smart techniques, which is basically human perception integrated into the encoder,” Thierry said, adding that his company’s EyeQ Content-Aware encoding has the ability to reduce bandwidth consumption by up to 50 percent.

“This is going to give you very high quality compared to the maximum rate you consume today. In Japan, for example, 85 MB/s for satellite could be brought down to something much lower using modern techniques like content-aware encoding.”

He said the current distribution method for 8K is to split up the content into four quadrants and process separate 4K images prior to delivery to the OTT service. This scheme was devised by NHK because four years ago in Japan nobody was able to encode 8K.

“Therefore, if you combine content-aware encoding, the cloud, and some smart engineering, you can do miracles with 8K,” Thierry said. “It’s all AI-based encoding and we're just at the beginning of what’s possible. We’re looking at 50 percent reduction compared with a classical encoding approach. Maybe we won’t need things like EVC going forward.”

AI-Powered Encoding

Among the AI-powered encoding approaches out there, researchers from Samsung Research’s Visual Technology team have developed AI ScaleNet, a codec based on AI that delivers 8K content on networks that typically support only 4K speeds, and all without the need for additional infrastructure. Amazon Prime Video is said to be experimenting with it now.

AI ScaleNet uses deep learning technology to minimize data loss during compression and enables 8K content to be streamed on networks with lower bandwidth capabilities. The 8K content is compressed to 4K quality using an AI downscaler and transmitted to the user’s TV. FOR-A Corp. (under the Fujitsu brand) is also working on this type of AI-assisted encoding, which is now being deployed by multichannel hosting and playout services like Brightcove and Kaltura. They are taking 8K mastered content and, with AI assistance, down-rezing to 4K for distribution. Then, using either metadata or encoding information in the stream, it smartly upscales in the 8K TV set.

Mandel said Samsung introduced AI ScaleNet at the 2019 CES and showed more at the Samsung Developer conference last October. It enables you to compress content at 35 MB/s for AV1 in 8K.

“The technology trains two deep neural networks to pick out the relevant portions of the content so that you can do a smart downscale and a smart upscale,” he said. “We train it in our labs against a set of images that work through the round trip of scaling and round trip of compression and all the different quality points. So, it actually takes a stack of neural networks to do all of this.

“So the content provider will put in 8K or 4K content and it’ll shrink the file size down in half,” he said. “So it will go from 8K to 4K or 4K to 2K. And then service providers can transmit that using existing infrastructures. So, with this technology you can put 8K content into a 4K pipe, or 4K into and 8K pipe. If you could only broadcast in HD before, now you can broadcast in 4K.”

8K Distribution Within Reach

On the TV side Samsung has implemented a smaller network in silicon on its new 8K TV’s that will do the reverse. So, when it gets a bit indicating that the downscale was done this way, it will undo it. And that network has been trained in conjunction with the downscaling network, so it’s smarter in how it paints in the detail.

“If you want to transmit at 50 MB/s HEVC, we feel very comfortable doing that at 25 MB/s for 8K, and conversely, with AV1, you can do something at 35 MB/s for 8K,” Mandel said. “This makes 8K a lot more reachable for distribution right now. And it also gives a bandwidth or quality boost to content providers that would want to use this in their normal flow. We recommend using it in all cases. The quality or just the downscale is at least as good as or better than some of the adaptive down scales that are in transcoders today.”

4K Upscaling

Harmonic’s Thierry suggested current 8K productions could shoot with three 8K cameras and down-scale the image prior to distribution. Upscaling to 8K would then occur at the TV set. If the quality is good, maybe 4K will be a reality on 8K TV sets without having 8K production.

“Harmonic was a proponent of this downscaling process in the early days of MPEG and MPEG never wanted to hear about the pairing of pre- and post-processing,” Thierry said. “I’m not saying we should not shoot in 8K, but there are a number of good alternatives that exist today that produce very good quality. This might be a good solution for the next few years.”

He pointed out that random tests have shown that 8K capture, even for 4K distribution, is much better than 4K native content.

“You see details better and you have much higher frequencies,” Thierry said. “We at Harmonic believe that 8K capture, even for 4K distribution, is a real plus. Can you call that an 8K service? I’m not sure. What’s important is that if we offer them an upconverted 4K image and tell them it’s 8K, the consumer better see an improvement in the picture they are watching.”

The panelists agreed with Thierry that while content providers like Amazon and YouTube don’t want to spend the extra money to shoot an 8K native production, they could offer 8K highlight channels of major sporting events.

“This could be done in 2021, but 2022 for the Olympics might be a better bet,” Thierry said. “I would bet World Cup 2022 highlights in 8K could be a viable service.”

“Finding a viable business model for an 8K service is perhaps the biggest hurdle thus far,” said Mandel. “Live content today is impossible to deliver to homes. But movies and on-demand content can be done very well. But do they have the motivation? That’s the question.”