Grace Hopper was one of the early pioneers of computing. She is quoted as saying "The nice thing about standards is that there are so many of them to choose from". The quote is also attributed to Professor Andrew Tanenbaum. Nevertheless, it is an accurate observation. There are a lot of standards!

We encounter standards everywhere, every day, from the power sockets on the wall to the way our mobile devices interact with one another. We should expect to see new standards describing how to interface with Artificial Intelligence (AI) and other smart systems in due course.

These articles focus on important standards for broadcast production and delivery in modern IP driven enterprises.

What Are Standards?

Standards describe the best practices for making compatible products from different manufacturers. For example, the dimensions of a connector and its pin connections, a data structure or an Applications Programming Interface (API). A standard could describe an entirely ephemeral entity such as a video stream or a file containing audio material.

Product manufacturing can be scaled up and distributed across many suppliers with standardized designs. This also facilitates spare parts provision and maintenance. The alternative would require every piece of equipment or software to be a bespoke and unique design. The costs of constantly re-engineering and re-inventing the same wheels every time would be prohibitive.

A Little Bit Of History

Standards have been around for a very long time. Much longer than you might think.  Here are some early examples:

Who Creates The Standards?

International standards such as those produced by ISO are the result of many industry experts collaborating through a process of consensus and approval. Each country supports its own committees covering specific areas of expertise feeding into ISO via a national standards organization. The standards are detailed and high quality but the preparation might take a long time.

Special interest organizations that operate internationally or regionally such as AES, SMPTE and EBU also collaborate to bring a high level of expertise to the process.

Smaller special interest groups (AIMS Media Alliance, AMWA and DTG) promote good practice with technical reports and recommendations.

Expertise is drawn from participating companies whose work informs the standards. A standard can be adopted informally by those companies to avoid any delay while it is being ratified.

Independent research laboratories and universities run projects to examine the viability of new technical approaches. The Fraunhofer Institute is a good example where the work they did on AAC audio compression facilitated the standards process.

Standards may evolve in an entirely different way by a large-scale adoption of proprietary technology. These de-facto standards are initially created by companies that make products to exploit them. Here are two examples:

• The Rich Text Format (RTF) document standard could only be used with Microsoft word processors at first. The format was easy to reverse engineer and is now widely supported by other applications. Microsoft have since published the specification for general use but it remains their intellectual property.
• Adobe PostScript code can be wrapped in a container to become a Portable Document Format (PDF) file. This has now been adopted by ISO as an international standard (ISO 32000) which is available free of charge.

A Taxonomy Of Standards

Organizing the standards into a taxonomy helps to see how they interact. Some large standards are divided into multiple parts, each of which might fall under different categories. This diagram illustrates our major areas of interest.

Video and audio standards are developed independently and integrated when a container describes how to embed and synchronize multiple tracks for streaming or broadcast transport.

Why Doesn't It Work Straight Out Of The Box?

Large and complex standards provide many alternatives for configuration with a lot of parameters each having a wide range of possible values.

Two companies might engage in building products based on a standard and intend that their products will integrate seamlessly with each other when they are deployed. If the standard allows any leeway for interpretation, these completely compliant products might not interoperate as expected if they use mutually exclusive implementations of the standard. Their customers cannot make them work together when they conduct the conformance testing.

This problem is caused by incomplete standards implementation on both sides. It is driven by pragmatism and budgetary controls on what is deemed to be a reasonable level of functionality.

Profiles & Levels

Complex technologies such as the AVC codecs introduced the concept of profiles and levels to simplify the interpretation of the standard.

An encoder does not always need to use all of the tools defined in the standard. Profiles describe a sub-set of the available coding tools that the encoder could use to compress the video. These profiles are designed around particular use cases to eliminate unnecessary features in the encoder implementation. The profile also provides hints to the decoder in the client player.

Levels define the picture size, frame-rate and bandwidth limits for the encoded bitstream to reduce complexity in the decoder.

Check that the profile and level support in your encoder settings is compatible with the decoder in your receiving device. Using higher profiles and levels may result in a better compression ratio but may also limit the range of devices that your content can be viewed on.

Standards Wars

Standards are sometimes weaponized for commercial gain. Perhaps by omitting support for a standard in a new product or by requiring customers to buy licenses to use certain features.

Organizations may build products that adhere very strictly to an accepted standard. Then they implement additional proprietary features and encourage developers to exploit them in their projects.

This serves several purposes. Firstly, they can rightly claim the moral high ground by declaring that they are wholly standards compliant - which is true. Once the proprietary extensions are deployed by developers, they are locked in to using that product. When a single maintenance release introduces a proprietary technology, removing it can take between 3 and 5 subsequent updates to replace it with your own solution.

The potential for using standards to lock customers into proprietary technologies still exists and needs to be anticipated to avoid it.

Glue-Ware

As an end-user faced with two incompatible products, you might still need to implement a work around and introduce bridging mechanisms.

Hardware interfacing devices to transform the signal from one format to another are widely used. For example, converting computer video output to a useful format so it can be integrated into the rest of the studio infrastructure.

IP based studios may still require a similar kind of virtual 'glue' to convert data streams or AV files from one format to another. The concept is similar but the implementation is quite different using apps and processes instead of hardware.

Classical hardware glue solutions tend to be instantaneous while software conversion may introduce some latency and delay the output somewhat. Hybrid solutions using the best of both approaches are also possible.

Overlapping Standards

Some regulatory bodies collaborate on specific standards topics. For example, CENELEC, ISO, BSI, EBU, ITU-T, JVT, MPEG and JPEG all share specifications for moving and still images. The output from these organizations leads to an ISO standard being published. Sometimes, a new standard is published and other organizations release essentially the same specification under a different name or number. The namespace is already crowded with acronyms and this can be confusing.

Risk Factors With Patents & Licensing

Check carefully whether a standard depends on patents before you commit to it. The authors may not have known which patents apply.

A useful standard might have patent licensing terms that make it very expensive to deploy. An open standards group might take the opportunity to create a compatible and patent-free alternative. This then becomes a de-facto standard that displaces the original. It is a major cost saving.

The standards development process tries to avoid this by not including patented technologies without an undertaking from the patent owner to license it for minimal costs to the implementer.

Submarine patents surface sometime after a standard has been widely adopted. This leads to a lot of unpleasantness. Be well prepared before committing to the use of a standard.

A potentially expensive situation arises where the patent holder attempts to monetize the usage (instead of the purchase) of equipment or software that includes their patent. This is a major problem for broadcasters who cannot be expected to pay a fee per viewer or per programme.

Careful architectural design can mitigate this risk factor. Web browsers such as Firefox rely on the operating system to provide support for playing AVC video. This is necessary because Firefox is an open source application with limited funding and the licensing fees are prohibitive. Delegating this to the OS means that only one shared license is necessary for all apps on that platform.

Conclusion

Standards are widely available but they are sometimes expensive to buy. They are often written in a deeply technical fashion which is hard to decipher. In the following articles we will drill down and examine the standards we depend on in broadcasting to unravel this complexity.

About The Standards Article Series

There are 26 parts planned for this series – due to publish throughout 2024. Here is a list of forthcoming articles with simple descriptions of their content:

Part 1 : An Introduction To Standards
The introduction and anchor article for the series covering the following topics: What are standards? Who creates them? Why do they exist? History, Benefits, Advantages, Disadvantages, Proprietary Extensions, and Profiles.

Part 2 : Standards For Broadcasting & Deployment
A section level article giving an overview of the standards relating to production and transmission or playout. This article prepares the ground for subsequent more detailed articles which will explore the following subject areas: ST 2110, higher bit rate codecs and profiles that are non-lossy, standards relating to exchanging material between different NLEs and Vƒx, and the AMWA workflow standards.

Part 3 : Standards For Video Coding
A section level article giving an overview of codec specifications. ISO and non-ISO standards will be covered alongside SMPTE 2110 elements to contextualise all the different video coding standard alternatives. Codec efficiency will be compared - AVC is roughly twice as efficient as MPEG-2 and HEVC is better still.

Part 4 : Standards For Media Container Files
A section level article giving an overview of files for transporting media essence. Certain kinds of files and video/audio formats go together but some also prohibit the use of other codecs. AVI and QuickTime are multimedia platforms as well as file containers and video codecs. Most of these are described by ISO standards but there are some de-facto proprietary formats that are important.

Part 5 : Standards For Audio Coding
A section level article giving an overview of MPEG & AES standards. The AES standards are referred to by ST 2110 which mainly defines how to use them. We will briefly cover AES 10 and 31 here.

Part 6 : About the ISO 14496 - MPEG-4 standard  
Here we describe all the different parts of this standard with a little of its history and coverage of the parts that are interesting but which have not gained any traction (such as BIFS). This article prepares the ground for subsequent more detailed articles where the important parts will be covered in greater depth, such as MPEG-4 Part 10 - AVC.

Part 7 : ST2110 - A Review Of The Current Standard
This is an outline of the different parts of this standard. How they use prior work to extend the SDI and related other SMPTE standards. As the series builds this article links to more detailed articles.

Part 8 : Standards For Designing & Building Workflows 
These are supporting standards that help with workflow construction and management. This is scaffolding for building out your workflows; AAF, MXF, NMOS etc.

Part 9 : Standards For On-air Broadcasting & Streaming Services
On-air broadcast and streaming service providers share many standards in common. There are some subtle differences in how they apply them.

Part 10 : Streams, Multiplexing & Embedding
Standards that describe video and audio compression only deal with single elementary streams. Audio needs to be embedded with video to make a viewable programme. Multiple channels need to be combined to make a transport stream for delivery over DSat, DCable and DTT, DVB & ATSC 1 & 3. Here there are opportunities to apply statistical multiplexing to use the available bandwidth efficiently when multiple channels are multiplexed together.

Part 11 : Streaming Video & Audio Over IP Networks
Delivering multiple streams on internet services viewed in web browsers is done quite differently to broadcast. This is covered by MPEG-4, Parts 8, 29, 31 and 33. MPEG-DASH. ST 2110 parts 20, 21 and 22. AES 3 & 67 are part of this as are HLS, SRT, RIST, RTP and RTSP protocols.

Part 12 : ST2110 Part 10 - System Level Timing & Control
How ST 2110 Part 10 describes transport, timing, error-protection and service descriptions relating to the individual essence streams delivered over the IP network using SDP, RTP, FEC & PTP.

Part 13 : Exploring MPEG4-Part 10 - H.264/AVC
The H.264/AVC codec has been very successful. Here we dig deeper into how profiles and levels work to facilitate deployment of delivery systems and receiving client-player designs.

Part 14 : About High Efficiency Video Coding (HEVC)
Here we look at the HEVC codec which is based on earlier work by MPEG on AVC and prior coding technologies. New techniques are employed to reduce the coded output size even further.

Part 15 : ST2110-2x - Video Coding Standards For Video Transport
SMPTE 2110 and its related standards help to construct workflows and broadcast systems. They coexist with standards from other organizations and incorporate them where necessary. In an earlier article we looked at the ST 2110 standard as a whole. This time we focus on the parts of ST 2110 that are concerned with video transport.

Part 16 : About MP3 Audio Coding & ID3 Metadata
The MP3 audio format has been around for thirty years and has been superseded by several other codecs – so here we discuss why it still has a very strong position in broadcast. We also discuss ID3 metadata tags which often accompany MP3 files.

Part 17 : About AAC Audio Coding
Advanced Audio Coding improves on the MP3 Perceptual Coding solution to achieve higher compression ratios and better playback quality.

Part 18 : High Efficiency And Other Advanced Audio Codecs
Our series on Standards moves on to discussion of advancements in AAC coding, alternative coders for special case scenarios, and their management within a consistent framework.

Part 19 : ST 2110-30/31 & AES Standards For Audio
Our series continues with the ST 2110-3x standards which deploy AES3 and AES67 digital audio in an IP networked studio. Many other AES standards are important as the foundations on which AES3 and AES67 are constructed.

Part 20 : ST 2110-4x Metadata Standards 
Our series continues with Metadata. It is the glue that connects all your media assets to each other and steers your workflow. You cannot find content in the library or manage your creative processes without it. Metadata can also control the end-user playback experience.

Part 21 : The MPEG, AES & Other Containers
Here we discuss how raw essence data needs to be serialized so it can be stored in media container files. We also describe the various media container file formats and their evolution.

Part 22 : AIFF files  COMING SOON
AIFF files contain raw audio samples that can be edited with audio workstation tools. There are a range of sample rates, track numbers and bit-depths for this data. It is an old format but it is actually a rather good solution for uncompressed audio, especially for archiving purposes.

Part 23 : MIME Types  COMING SOON
This standard is important because it defines how a receiving client application will treat the incoming content whether it is delivered in a container file or as a stream. Your workflow will likely use this information to control how assets are processed. It is an important piece of metadata. The standards are managed by the IETF as RFC documents. This was originally defined for mail attachments in RFC 2045 but it is also used in Web Server response headers when delivering downloads. Since that technology is also used within a production environment based on IP, it becomes important in that context too.

Part 24 : Timed Text & Subtitles & ST 2110-43 COMING SOON
MPEG-4 parts 17 and 30 describe synchronised text streams. Similar coverage is described in SMPTE ST 2110 parts 40 and 43 all running on an ST 2110-part 10 foundation for timing and control. This is reflected into web-based media players as VTT tracks embedded in < track > tags that are contained within < video > and < audio > tags in HTML-5 web pages. A timed text item, triggers a JavaScript event which can be trapped by an event handler to call some active code to action. This can alter the appearance and content of the user interface or simply present a sub-title text. Visual material can be triggered for presentation within strictly audio presentations which blurs the distinction between ‘TV’ and ‘Radio’ like user experiences. There isn’t really any difference from a technical point of view other than the amount of bandwidth required for delivery.

Part 25 : Client-side Players  COMING DURING 2024
Building client-side web browser-based players is not just for adding A/V to web pages. It is also used to create entire user experiences in TV receivers and set top boxes. There are a variety of alternative approaches which are becoming de-facto standards. Some are built as apps using Java or Android environments. Others are proprietary such as Apple i-Devices and TV platforms. Smart TV sets support a variety of HTML-5 or TVML authoring environments. Delivery of these user experiences tends to be via a proprietary conduit either over the air or via an IP connection. Wired vs. WiFi also becomes an issue for performance, QoS and reliability.

Part 26 : Other Standards  COMING DURING 2024
An overview of some of the additional standards covering content management, interactive media and internet protocols which might be of value to broadcasters.

These Appendix articles contain additional information you may find useful: