The finished size of the media content directly affects the cost of transmission. Broadcasters want to fit more channels into a multiplex and streaming services want to deliver more on-demand content within their available bandwidth.

international standards have historically been driven by MPEG. Proprietary standards are popular with streaming services and the orgs that sell technology to them. Open-source codecs are gaining traction because they are generally patent free.

Standardized deployment codecs deliver video to end users via players or streaming TV apps. Codecs for production workflows tend to be proprietary and based on the camera output or the editing and visual effects pipeline tools. We will look at those in a later article.

Video Is Becoming More Pervasive

Digital signage and inexpensive large flat screens facilitate the deployment of video advertising everywhere. We see examples in shops, on public transport and on entertainment systems in cars and many other places. Examples of current and future applications are:

• TV Broadcasting.
• Removable media (DVD & Blu-ray).
• Streaming services (Netflix, Disney+, AppleTV+ Prime etc).
• Digital signage for shops, billboards and public transport.
• Automotive and self-driving solutions.
• Autonomous vehicles for shopping delivery.
• Long distance drone delivery systems.
• Augmented reality.
• Virtual reality (Metaverse).
• Security monitoring.
• Machine Vision.
• Consumer products.
• Toys.
• Gaming.

Current & Future Deployment Codecs

Historically, most codec specifications were described by ISO standards based on the MPEG working groups research. There have always been proprietary standards used in closed streaming services where the provider controls the choice of player. Open-source patent-free codecs are rapidly gaining traction and challenge the status quo because they are commercially attractive. This table lists the most popular codecs:

ISO MPEG Video Codec Evolution

This diagram shows how the ISO MPEG codecs have evolved from a common ancestor. The EVC codec is a new initiative based on a patent free toolkit. The LCEVC codec is designed to extend any underlying codec. The enhancement layers improve the resulting decompressed picture quality. The VP9 and AV1 codecs are developed independently but can be enhanced with LCEVC.

There are variants of the AVC codec to support stereographic 3D viewing but this has fallen out of favour now. There are online resources with more details about MVC, MVCD and 3D-AVC should you need them.

Obtaining The Correct ISO Documents

There are many ISO standards documents relating to video coding, encapsulations and carriage. I found more than 230 individual ISO documents (parts of standards) as I researched this article. You only need to obtain the pertinent ones for any project but many of them are cross-referenced and you may need to read others to understand the full picture:

• They describe what you need to build.
• How to build it.
• Why you should do it.
• How to combine other components into a higher-level construct.

The definitive source for standards is the ISO online store. The latest released version will be available. Early drafts of new versions are sometimes on offer and ISO also releases free Publicly Available Standards (PAS). The standards are divided into separate parts that deal with different topics and these need to be obtained individually. There may be several amendments since the most recent edition. They are periodically edited into the main standard for the next published edition.

Use this notation for citing references within the ISO standards:

ISO {nnnnn} - Part {pp} : Year {yyyy} - {Sub-section or appendix}

Locating the specification for the video codec you want to learn about is not always easy. Sometimes the codec will be described in a standard all of its own. More often they are one of the parts of another standard that is not obviously about video coding. The VVC codec is described as part 3 of the MPEG-I standard.

NOTE: Descriptions of Scalable Video Coding (SVC) refer to Annex G of the ITU-T H.264/AVC recommendation. The ISO standard with substantially the same specification, moves it to annexe F of ISO 14496 part 10. The Multi View Coding details are likewise transposed.

ISO Standardized Video Codecs

Here are the currently relevant ISO standards for video coding technologies. Use the name column or standard and part reference to search for supporting materials. MPEG provides useful insights and uses the name as a key. ISO has a web page for each standard.

Looking Forward To New ISO Codecs

New and interesting video coding concepts are emerging as we look forward to VR and other future applications. Projecting moving video into a 3-Dimensional space is already happening. To succeed commercially it needs to be delivered consistently so it is portable to many different headsets and players. There are also some interesting developments that allow existing codecs to be enhanced to improve viewing quality.

Patents & Licensing

The ISO and MPEG codecs contain patented technologies which mandate license fees. The license potentially affects broadcasters, consumer products manufacturers and streaming companies in different ways. Streaming services will find a license free alternative far more attractive.

Proprietary Video Codecs

Proprietary and open-source solutions are an alternative to international standards-based codecs. Any license fees (if there are any) would be embedded in a service contract through a single provider.

Proprietary standards tend to be described in a single document available free of charge rather than the expensive multiple parts of an ISO specification.

Conclusion

Your choice of codec is most likely driven by the cost of licensing and the level of support in the target client viewing devices. License free open-source codecs that are widely available in web-browsers are very quickly becoming dominant.

If you must use ISO standards then AVC is optimal for standard-definition. HEVC would perform well on higher definition and VVC will outperform both. All three require royalty payments.

If a royalty-free solution is what you need then AV1 appears to be the strongest contender. Apple has built an AV1 hardware decoder into their latest M3 CPU chips. This will likely migrate into the A series chips in iPads and phones. The leverage from this virtually guarantees its dominance in the market.

These Appendix articles contain additional information you may find useful: