The analogue television switch-off has been completed in many developed countries over the past few years, and the availability of more digital broadcast outlets has paved the way to even more immersive audio and video formats. The broadcast industry is on its way to all-HD. But increasing demand for enhanced end-user experiences with high-fidelity content (especially for home theatre and premium events like sports) is rising. End users are calling for Beyond HD.

In parallel, with the recent availability of 4K-UHD (Ultra-High Definition) screens (TVs, tablets, and mobiles) on the market, the end user remains with a new offer of high display-capacity devices, but is facing an almost non-existent associated service offer. Some first new services experiments were conducted and demonstrated this year. Despite the “availability” of new devices, a massive deployment of real services is not feasible through the current end-to-end delivery chain because of bandwidth limitations. Video compression must be more efficient to enable the delivery of such Beyond HD services.

The first question is, do we really need to go Beyond HD? What benefits are gained, and what are the implications of Beyond HD? Can new technology improve the consumer viewing experience while not requiring a bandwidth increase? Broadcasters’ and operators’ objectives remain the same over time: improve the TV viewing experience, reach more customers and deliver new services offers in the same or less bandwidth to reduce costs.

After reviewing the existing leverages to provide an enhanced viewing experience, we will propose a description of Beyond HD facets and describe the current limitations to broad deployment of these new services. Then we will give a brief description of HEVC (High Efficiency Video Coding), the next-generation video compression standard, with latest results demonstrating its direct benefits for Beyond HD services. Finally, we explain why the HEVC technology will ease the “all progressive” broadcast chain while unlocking the development of the next enhanced services generation, without thereby announcing farewell to “interlaced” mode.

New trends: from capture to devices

In very large measure, today’s broadcast television production chain still operates in interlaced mode. With the advent of digital, the latest generation of cameras captures in progressive mode.Nevertheless, most still output an interlaced feed, as the whole progressive broadcast chain is not yet ready. We estimate that 70 percent of the worldwide HD content production is still done in interlaced mode (i.e.1080i25/30) and only 30 percent in the progressive (i.e. 720p50/60). Some market precursors have already undertaken initiatives to get the whole production chain in 1080p50/60.

Devices with enhanced display capabilities, increased decoding capacity (e.g. multicore architectures) and the ability to manage higher spatial resolution, frame rate and bitrate are about to arrive on the market. New UHD “big screen” devices are well on the way, but we will also see premium HD services soon reach tablets, mobiles and OTT devices owners. These services may well reach a mass market much faster on the “small screens,” as consumers around the world are increasingly driven to upgrade their phones and tablets on two-year cycles. We believe that end users will ask for an enhanced viewing experience on all these screens. With the development of “TV Everywhere,” the market may soon call for Beyond HD.

The access to these higher forms of premium content and new enhanced services is currently prohibited by the bandwidth limitations on managed and unmanaged networks. The development of new infrastructure like 4G and/or standards like DVB-S2/T2 could (partially) circumvent these limitations and free the access to Beyond HD services.

What is Beyond HD?

Beyond HD is the next-generation TV systems that will offer an enhanced end-user experience through new audio-visual technologies to provide a more realistic experience, stunningly better image quality and more accurate visual details as compared with today’s current HDTV systems.

Keys to improving the viewing experience

By commonly associating Beyond HD with 4K-UHD, end users and TV suppliers narrow the focus on improvements to spatial resolution only. But Beyond HD is not only a matter of spatial resolution. Improving the end-user viewing experience can be realized through other characteristics of the video, including frame rate and the depth and realism of color displayed.

The choice of a preferred frame rate has been a hot topic of debate for many years. It has been clearly demonstrated that 60Hz is better than 30Hz for fast motion content, but several studies indicate that frame rates even higher than 100Hz are needed to avoid temporal artifacts. During IBC 2013, EBU presented the results of the HFR (High Frame Rate) project, which demonstrated the increase in viewing comfort observed from 60Hz to 120Hz, but also from 120Hz to 240Hz.

In the production of HDTV programs, all the colors captured using video cameras are first defined within the gamut formed from the CRT-centric reference RGB-primaries. With the arrival of new displays’ technologies (e.g. LCD, LED) as well as new cameras with extended characteristics, colorimetric becomes of major importance to improve the user experience. First wide color gamut can be considered, as in BT.2020 [4], which covers 75.8 percent of the CIE 1931 color space whereas the BT.709 [3] is only covering 35.9 percent. Bit depths of 8,10,12,16 bits are also important parameters to control the accuracy of the color sampling. Chroma sampling rates also need to be considered. Finally, a High Dynamic Range (HDR) would allow for a more realistic perception between the brightest and darkest elements in a scene.

Authors would like to add unusual characteristics to this set of parameters implied in the end-user viewing experience: the Film Grain. Beyond HD and premium services for home theatre would take into account a characteristic that is finely tuned by film directors. Film grain plays a significant role in the atmosphere of a scene. It was observed that, in terms of visual perception, film grain reinforces the feeling of realism for the audience.

Various approaches, whether proprietary or standardized, are considered in order to deal with extended color management. However, whether this is based on full or scalable metadata, higher bit depths, higher frame rates, or enhanced spatial resolutions, such choices will increase the amount of data to be processed and raises important technical and bandwidth issues all along the production and the broadcast delivery chain.

Beyond HD services

Beyond HD can have different facets and provide different benefits depending on the parameter sets applied to the images.

HD+ (1080p50/60)

Experiments have demonstrated that content with fast and large camera motion requires enhanced viewing comfort. Sports content or live event viewing experience challenges today’s HDTV with image fluidity and motion quality. That may result in an unsatisfying experience for the end-user, especially as screen sizes increase. Moving from 1080i25/30 to 1080p50/60 production and distribution can enhance the end-user viewing comfort but of course requires a doubling of the information to be displayed. Hence, we can definitely consider a 1080p50/60 service as a Beyond HD service. We propose to call it HD+ in the sequel. Content like sports, theatre, and live concerts may be good candidates for upcoming HD+ services. Is there also a potential need for what we might call HD++ services with even higher frame rates (e.g. 120Hz), alternate color spaces and higher dynamic range?

Nevertheless, we will keep in mind that broadcasting a feed at higher frame rates (50/60, 100/120 or even higher) will definitely lead to increases in the required bandwidth. Would doubling the frame rate require additional bandwidth in nearly similar proportions?

UHDTV

A better sense of realism, immediacy and intimacy with the viewing experience are definitely keys to television success. Reaching into the emotions of the audience is essential. UHD provides additional features that can touch upon all these emotional aspects by increasing the fidelity of image detail and movement, increasing viewer involvement and providing an enhanced sense of “being there” (See Figure 1).

As discussed earlier, UHDTV is not only a matter of spatial resolution but also a matter of wide color gamut, higher dynamic range and higher frame rate, together enabling a radical improvement in viewing experience. In this context, the ITU standardized UHDTV (as 3840×2160 pixels) in [4]. However, this standard does not clarify which subset of features should be used for broadcast applications. We believe that early UHDTV broadcast would be performed with 3840x2160 pixels pictures, assuming a 10 bits signal transmitted at 59.94Hz, with multi-channel audio up to eight channels. Would UHDTV fit on current distribution architecture?

Very first Beyond HD experiences and limitations

UHD experimentations on satellite

During this calendar year, satellite operators the world over (Europe, North America and Asia) experimented with UHD transmission. Most of them experienced the satellite broadcast of Quad HD channels on their transponders. As an example, broadcasting via DVB-S2 a UHD/Quad HD channel encoded in H.264/MPEG-4 AVC required at least 40Mbps in order to provide the video quality expected with UHD. Limitations are reached; knowing that, practically, a transponder broadcasts via DVB-S2 at around 70 Mbps, only 1 UHD/Quad HD channel would fit.

If the first UHD offline encoders are arriving on the market, high-power processing required for real-time encoding prevents UHD Live (i.e. single Tile) deployment any time soon. The only solution tried so far for live is to broadcast Quad HD content produced by four encoders fed with 1080p50/60 on 3G-SDI links. Then, four synchronized IRDs are needed to decode those streams and four links (3G-SDI or HDMI) are finally used to transport the uncompressed signal to the display. Even if such architecture can be considered for contribution matter, bandwidth constraints remain very important and neither the set-up (i.e. four decoders or four STBs) nor the bandwidth is realistic for distribution matter. Alternative solutions have to be considered.

VOD content providers launch a first 4K-UHDTV service

Beginning in September 2013, a market actor launched a 4K-UHD video “store.” This service is a world’s first and is not likely to be broadly deployed, as there are limitations due to the use of a proprietary UHD TV set and an associated proprietary Media Player. To launch such a UHDTV video offer, independent Video On Demand (VOD) providers, broadcasters or operators would need to provide their UHD first-mover customers new equipment since current legacy STB, PCs, game boxes, etc. do not support UHD decoding

With the current compression technology, such services favor end users having high-bandwidth network since the download duration may lead to a limited usage and/or a limited audience.

HEVC: New-generation video compression

Several solutions exist to address network congestion issues and bandwidth constraints: from infrastructure evolution (e.g. DVB-T/S to DVB-T2/S2 standards) to compression technology evolution. Thus, the new generation video compression standard HEVC (High Efficiency Video Coding) has been developed by the JCT-VC—Joint Collaborative Team Video Coding—a joint working group of both ISO/IEC Moving Picture Experts Group (MPEG) and ITU-T Video Coding Experts Group (VCEG). This new standard was finalized in January 2013.

On performance

History is once again repeating itself; ten years after the 50 percent coding gain provided by H.264/MPEG-4 AVC over MPEG-2, yet another 50 percent improvement from HEVC over H.264/MPEG-4 AVC has been attained. The latest objective (PSNR-Bjontegaard) and subjective (Mean Opinion Score) tests [7][8], on various broadcast content between HEVC and the AVC High Profile, have already shown a gain in compression better than 40 percent for HD content. Subjective tests indicate that the HM encoder (i.e. HEVC reference software) is saving at least 50 percent bitrate compared to the JM (H.264/MPEG-4 AVC reference software) for equivalent perceived quality.

Similar performance has been exhibited on a large set of content using both our H.264/MPEG-4 AVC and HEVC encoders. Moreover, in the context of the 4EVER project, we conducted a wide study considering both scanned and native 4K-UHD content presented in which exhibited coding gains in a range of 38 percent to 50 percent based on subjective measurements.

4EVER is a French consortium supported by Europe (FEDER), French Ministry of Industry, French Regions of Brittany, Ile-de-France and Provence-Alpes-Cote-d’Azur and related Competitively Clusters. It is 3-year collaborative R&D project running from 2012 to 2015

Yet another hybrid codec?

Despite similar overall architecture, the HEVC standard contains a set of advanced tools that together make this new codec more flexible and much more efficient than H.264/MPEG-4 AVC. This codec is particularly tailored toward high-resolution images with block sizes that could be as large as 64x64, and high frame rates with advanced motion vector prediction (See Figure 2).

On profiles

Unlike H.264/MPEG-4 AVC, the goal with HEVC was to consider only a reduced number of profiles. Thus (so far) only three profiles have been defined: Main, Main 10 and Still Picture. The Main profile covers the wide set of current consumer distribution applications from mobile devices to HDTV and supports an eight bits 4:2:0 signal. The Main 10 profile targets premium and Beyond HD applications by offering a 10 bits support, therefore suitable for contribution and “high end” use cases. Finally, the Still Picture profile concerns I pictures only.

Unofficially, the current HEVC standard is called “HEVC v1” since different research work related to further extensions/amendments has been undertaken with finalization planned for early 2014. In addition to the Scalable and the Multi-view extensions, the Range Extension (Rext) is of major importance for the purpose of professional and Beyond HD applications. Indeed, these extensions not only target bit depths of 10 bits or higher, they look to provide for 4:2:2 and 4:4:4 chrominance sampling, but also a wider color gamut. HEVC is being considered as the next generation’s premium compression technology. Broadcasters, Movie Studios and other high-end content producers are closely following these activities.

HEVC performance for Beyond HD

In order to assess the performance of HEVC on Beyond HD content, we performed a detailed study and the main results are reported here. Our main objectives were to evaluate the impact of frame rate, spatial resolution and bit depths on the compression efficiency of HEVC. All the reported results are based on objective quality measurement (PSNR).

When comparing HEVC and H.264/MPEG-4 AVC performance on Beyond HD content like 1080p50/60 or 2160p60 sequences, we observed a compression gain of between 38 percent and 45 percent. The results illustrated on Figure 3 have been validated on a large set of sequences.

In order to evaluate the impact of the increase of frame rate on the bitrate, we investigated various video sequences captured in 50/60 Hz and 120Hz with different motion and texture characteristics. Figure 4 illustrates some results obtained on 1080p and 2160p sequences. A bit rate increase between 19 percent and 42 percent is observed when going from 25/30Hz to 50/60Hz, while a lower bitrate increase between 17 percent and 28 percent is observed when comparing 60Hz to 120Hz. Note that the impact of the spatial resolution seems negligible on this percentage.

Broadcast quality

First results provided in [9] indicated that the HEVC coding gain would allow a “broadcast quality” of UHDTV at video bitrates below 13Mbps. These results were obtained with the HEVC Main profile using 4:2:0 8 bits for 60Hz content and have been confirmed across a wide set of eight bits and 10 bits video sequences from 24Hz to 60Hz, leading to our finding that “broadcast quality” for UHDTV with HEVC can be achieved at bitrates between 11Mbps and 18Mbps.

The Main 10 profile

After confirming that bitrate savings between H.264/MPEG-4 AVC and HEVC are similar for both Main and Main 10 profiles, we performed some comparisons between various 10 bits sequences. A bitrate saving in average of 5 percent is observed for Main 10 versus Main, confirming the relevance (if needed) of such a 10 bits profile for the Beyond HD experience.

On film grain

One contrary result of the good performance of using HEVC for UHD content is the impact on film grain. Indeed, within the range of bitrates between 10 and 20 Mbps, the “desirable” grain of film-based sources is identified as noise and tends to be eliminated from the signal, which of course may not be what the director intended. This is why we recommend the use of the HEVC Film Grain Characteristic SEI message to allow the addition of controlled “grain” on the decoder/display side.

Why is HEVC the key to unlock the Beyond HD broadcast chain?

Beyond HD services will definitely provide end users with an enhanced viewing experience. However, so far, bandwidth limitations prevent any massive and commercially viable deployments in the current infrastructure.

With the arrival of HEVC, as demonstrated in the previous results section, Beyond HD services will become a reality. With the emancipation of “TV Everywhere” usage, the increasing penetration of OTT devices and the rise of high video quality expectations, Beyond HD services in concert with HEVC compression could be deployed with only a very limited (or even no) impact on the bandwidth costs for broadcasters, programmers and network operators.

Thus, services like HD+ (1080p50/60) could be delivered at bitrates slightly below today’s 1080i data rates with H.264/MPEG-4 AVC. Higher bit depth and/or frame rate signals (e.g. 10 bits, 120Hz) could even be broadcast at similar bitrates as those used today for transmitting HDTV in MPEG-2. UHDTV deployments of services at 60Hz with 10 bits signal could be envisioned at bitrates between 11 and 18 Mbps. Using HEVC, up to 4 UHD channels could fit on a satellite transponder (DVB-S2). Benefits brought by the combination of Beyond HD and HEVC (i.e. improved end-user experience and increased number of services) should be monetizable!

HEVC would enable the emancipation of the Beyond HD formats. With such ideal “meta-formats,” all “sub-formats” for all screens can be easily addressed. In the context of multi-screen, using such progressive formats instead of 1080i25/30 would simplify the broadcast chain deployment, while limiting and avoiding complex processing steps like de-interlacing etc.

Is this one more step towards the “progressive only” broadcast chain? 

HEVC does not mean “interlaced end of life”

About 70 percent of the HD television broadcast around the globe is interlaced, most specifically 1080i25/30. Moreover standard definition SD (i.e. 480i or 576i) is only broadcast as an interlaced format. The legacy interlace format is still the most dominant TV format in the world.

As described above, HEVC offers very attractive compression benefits but lacks any specific tools designed for interlaced formats. Indeed, unlike the efficient PAFF (Picture Adaptive Frame-Field) and MBAFF (Macroblock Adaptive Frame-Field) coding tools of H.264/MPEG-4 AVC, HEVC was originally designed for progressive applications only. One may agree that ultimately the switch from interlace to progressive formats for all of television is simply a matter of time, but the time is not now. There are many technical, operational and commercial challenges for today’s interlace “houses” to solve before they can willingly make a move to progressive. Agreements have to be obtained from content producers/owners; many of them categorically refuse to see their interlaced content modified into progressive. 

Interlace coding: objective and subjective results

As co-chair of the HEVC working group on interlaced coding, we have made various contributions in collaboration with renowned television experts from across the globe, leading to the integration of new tools within the HEVC standards to allow for full support of interlaced coding. New SEI and VUI messages were introduced. Even without dedicated compression tools, HEVC makes it possible to compress interlaced content either as Frame coding (progressive only) or as Field coding (field only at a doubled frame rate).

Objective and subjective measurements were performed on legacy interlaced content. As only HD content was considered in the MPEG working group, we decided to evaluate HEVC using the set of broadcast content used to evaluate MPEG-2 and H.264/MPEG-4 AVC compression technologies. Comparisons were performed using the JM18.5 and the HM12 software.

As illustrated on Figure 5, HEVC objective bitrate gains on legacy interlaced content are found to be around 30 percent, both in SD and HD. Hence for instance, SD could be broadcast at 1.25Mbps instead of 1.8Mbps and HD at 5.5Mbps instead of 8Mbps. In addition, the 4EVER Consortium performed a subjective evaluation of HEVC on typical 1080i content reported in [10]. The conclusion of this study is that a subjective coding gain of about 50 percent is observed assuming the same perceived video quality.

Interlace coding is still alive

Such results show that the HEVC standard allows for the efficient coding of interlaced content. Thus, legacy television applications could benefit of HEVC by lowering the bitrate by at least 30 percent (or alternatively, increasing video quality). Following this conclusion, in September 2013 the DVB (Digital Video Broadcasting) group decided to integrate the support for HEVC interlace coding. We shall see interlacing for quite some time still in the broadcast industry.

What should speed up the arrival of Beyond HD to the market?

First-mover programmers, broadcasters, or video platform providers may use the occasion of upcoming sporting events to accelerate the market delivery of Beyond HD: the Asian Games in Korea in the fall of 2014, and the Rugby World Cup from the UK in 2015 are likely targets. These events would be fantastic laboratories for testing the latest service innovations, and in a competitive landscape, operators and programmers should play the innovation card to market a differentiated offer.

The two ends of the Beyond HD broadcast chain, from cameras to TV sets and video projectors, have been in limited availability for almost a year and are starting to become affordable. However, native UHD support for most other devices like switchers, encoders and STBs is still expected. Moreover, the post-production workflow requires adaptation to higher bitrates, process and storage capacities; the first challenge in deploying Beyond HD is in baseband.

Fortunately, the technical landscape evolves very quickly, and some of these technical limitations are beginning to fade away. Despite the announcements of equipment and standards like 6G or 12G-SDI, the natural trend for producers, operators and broadcasters (with long-term desires but short-term capital buying cycles) is to work around the bottlenecks by using what they already have in new ways, like aggregating multiple 3G-SDI links.

On the end-user side, the arrival of HDMI 2.0 (with a support of 4Kp60 up to 10bits) could de facto set up the UHDTV broadcast format Phase 1. Extended gamut, HDR and HFR should arrive in a Phase 2, but when? Regarding decoder building blocks, one can expect the arrival on the market early 2014 of STB supporting UHD services. But we believe that the upcoming “Smart TVs” able to natively manage UHD will definitely speed up the arrival of UHD services.

In addition, infrastructure evolution, like DVB-S2 extension, is planned before the end of the year and can increase the spectral efficiency from 30 percent to 50 percent, enabling users to transmit even more or better signals from satellite at higher bitrates.

Conclusion

After describing the various levers that can enable the Beyond HD viewing experience, potential future (premium) services like HD+ (or HD++) have been discussed that could benefit television users in the very near term.But we also discussed the implications of the new immersive television experience offered by the promise of UHDTV.

New test results provided in this paper demonstrate HEVC’s high compression performance and its direct benefits for Beyond HD services deployment. Finally, we explain why this compression technology can ease the “all progressive” broadcast chain deployment and why it will not lead to the premature end of life for “interlacing.”

Despite the fact that we are convinced that both HD+ and UHD bring real enhanced viewing experiences, the market, to date, seems completely consumed with the prospects and promises that UHD offers. Indeed, UHDTV appears to be easier to market to the end user. The UHDTV value proposition is based on “real’ differentiating elements: new TVs offer larger screens, new immersive experiences, and forms of new premium content for home theatre, live concerts and sporting events. UHDTV should be monetizable!

Even if the HEVC compression technology will unlock the development of the next enhanced services generation without requiring any (major) infrastructure modification, the availability of HEVC Beyond HD encoders and decoders remains the question. Thus, HEVC offline encoders for UHD content have just been made available, but what about professional HEVC live UHD encoders?

About the authors:

Sophie Percheron has been Product Marketing Manager for Live Distribution at ATEME since October 2012. Sophie is responsible for building comprehensive solutions that deliver linear channels over Cable, IPTV, Satellite, Terrestrial as well as Over-The-Top networks, enhancing the TV experience with value added services. In her role, Sophie drives the growth of the Live Distribution solution and related product roadmap definition to address today’s and tomorrow’s challenges in the Broadcast & Broadband Live distribution markets. Before joining ATEME, Sophie worked at Bouygues Telecom and Microsoft as Product Marketing Manager and Senior Program Manager positions. Sophie holds a Master Degree in Marketing from Paris Dauphine.

Jérôme Viéron received his Ph.D. in computer science from Rennes  University in 1999. He is an active contributor to standardization efforts led by the International Organization for Standardization (ISO) and the International Telecommunications Union -Standardization (ITU-T) groups. He was very active in the H264/MPEG-4 SVC standardization process. In 2007, he joined the Video Processing and Perception Lab of Technicolor R&I as senior scientist exploring new technologies for future video coding applications and standards. He joined ATEME in 2011, as advanced research manager. Viéron is in charge of French and European collaborative R&D projects and works on new generation video coding technologies. He is an active contributor to the standardization process of high efficiency video coding (HEVC) and is implied in the 4EVER (for Enhanced Video ExpeRience) consortium which aims at researching, developing and promoting an enhanced Television Experience, using HEVC as well as ultra-HD.