Standards: Part 4 - Standards For Media Container Files

This article describes the various codecs in common use and their symbiotic relationship to the media container files which are essential when it comes to packaging the resulting content for storage or delivery.


This article is part of our growing series on Standards.
There is an overview of all 26 articles in Part 1 -  An Introduction To Standards.


Media container files are as the name suggests, a means by which you can store your media in a container file for ease of production and deployment. They also enable the addition of time-synchronised text to support subtitles. The text can include mark-up to add styling during presentation. Special cases such as URL tracks use plain-text but are marked for special treatment in the player. Containers can also include descriptive metadata to facilitate the production.

What Do We Need To Carry?

Modern containers can carry multiple video streams and many audio tracks for language, accessibility support and surround sound. Other media types can be stored and synchronised with the video and audio for subtitles and interactive presentations.

Content Notes
Video Lossless for production, lossy for deployment.
Audio High quality for production but can be down-sampled for deployment.
Audio description Additional audio track for accessibility.
Surround-sound Multiple tracks of spatial audio.
Subtitles Stored as Unicode text in one or more languages.
Metadata Describes the format and semantics of the content.

 

In due course, other kinds of data will be needed to support new formats like graphical point clouds and 3D spatial information for integration with the Metaverse.

Containers Vs. Codecs

Confusion arises because the containers sometimes have the same name as a codec stored inside them. The H.264 codec has several alternative names which adds to the uncertainty.

Currently , the most popular codecs for end-user deployment are:

  • AV1
  • AVC
  • HEVC
  • MPEG-2
  • VP9

These can all be carried in the ISO standard MP4 container along with anything else that has been registered as a compatible media type. Virtually anything can be stored in a Matroška container file.


Sometimes you will see containers described as being lossless but the containers themselves do not affect the video quality. They are simply a means of packaging the output of a codec. It is the codec that might be lossy and not the container!


Base Media File Formats

Base Media File Formats are foundations that provide a basic structure for storing arbitrary data of any kind. Other container formats are derived from them either by extending them or by using profiles.

Format Description
ASF The Advanced Systems Format (ASF) facilitates Windows Media content delivery.
MKV The Matroška format (pronounced Matroshka) is used as the basis for WebM which only needs a sub-set of its functionality. It is named after the nested wooden dolls from Russia which illustrate the internal storage structure. The profiles dictate which codecs can be used.
RIFF The Resource Interchange File Format is the basis for the AIFF files that carry extracted CD audio.
ISOBMFF Described in MPEG-4 Part 12 and extended in MPEG-4 Part 15.

 

The most versatile container formats are the ISO Base Media Format Files (ISOBMFF) also known as MP4 (MPEG) and the MKV (Matroška) format. These offer the widest range of codec compatibility and support across multiple platforms. The MP4 format in particular is well supported in all browsers and many other kinds of devices. The AVI file format is almost as versatile but lacks support for some modern codecs.

Popular Container Types

These are the most popular container types built from the base media formats:

Format Description
WebM Web Media Files were designed by Google and are royalty free. They are optimised for use in HTML5 web pages and supported by the latest web browser versions. Based on a profile of the Matroška base media format, WebM only carries AV1, VP8 and VP9 video codecs. Audio is compressed with Opus or Vorbis codecs.
MKV The Matroška base media format supports a variety of profiles for different applications. It is currently the only container format that supports Multi-View-Coding (MVC). This container could be very useful in an automated workflow. Although it is a foundational format, it can be used directly and bounded with profiles.
MP4 MPEG-4 extends the ISO Base Media Format to support additional codecs.
QTFF QuickTime files are usually stored in MOV files. Modern apps use the MPEG-4 file format instead. Consequently, QuickTime is gradually fading into history. It is important as a container for Apple ProRes coded video.
ASF The Advanced System Format is useful for moving content around production workflows and is partially supported inside MXF containers.
AVI The Audio Video Interleave files are a versatile container format based on the RIFF file structures. AVI is useful for archival purposes although it is becoming less popular as new container formats evolve to replace it.
MXF SMPTE 377M describes the Material Exchange Format files used in production environments. This is the only alternative to QuickTime for carrying ProRes video content. The specification includes metadata support and some compatibility with Advanced Authoring Format (AAF) tools.

 


Valuable archive footage may have been stored in a deprecated format. Keeping old hardware running with ancient OS versions allows you to run a conversion process to extract the legacy material for repurposing.


ISO MPEG-4 Containers

The ISO Base Media File Format is based on the same DNA as the Apple MOV file format developed for QuickTime. Apple designed this file structure to support multiple simultaneous media tracks of any type.

Combine MPEG-4 Parts 12, 14 and 15 and RFC 6381 to fully understand the MP4 container file structure. Then examine the MP4 registry to find descriptions of the internal chunk storage ATOMs.

Standard Description
MPEG-4 Part 1:2010 MPEG-4 Part 1 describes the version 1 storage format. That is obsoleted by Part 14. The rest of the Part 1 standard is undergoing revision.
MPEG-4 Part 12:2022 ISO Base Media File Format (ISOBMFF) is technically identical to the JPEG 2000 file storage defined in ISO 15444 Part 12 which it replaces. A new revision is under development.
MPEG-4 Part 14:2020 MPEG-4 file format version 2 completely replaces the specification in Part 1 and is based on Part 12.
MPEG-4 Part 15:2022 Describes how to package and store MPEG-4 Part 10 AVC and HEVC video. It is based on Part 12. Amendments have been published.
IETF RFC 6381 Specifies how supported media types are described in the metadata.
MP4 ATOM registry The coded content is stored as structured chunks of data called ATOMs. A complete list of the ATOM types is available from the MP4 Registration Authority.
(https://mp4ra.org/registered-types/boxes).

 

Containers Vs. Compatible Codecs

Certain combinations of video and audio codecs are frequently used together. If you defy the conventions and use an unusual combination, your viewers may only see the video but not hear the sound, or vice-versa. The container type dictates your choice of audio and video codecs.

Format WebM MKV MP4 QTFF ASF AVI MXF
MPEG-1 Video No Yes Yes Yes Yes Yes Yes
MPEG-2 Video No Yes Yes Yes Yes Yes Yes
MPEG-4 Part 2 Visual No Yes Yes Partial Yes Yes Yes
AVC (H.264) No Yes Yes Yes Yes Yes Yes
HEVC (H.265) No Yes Yes Yes Yes Yes Yes
AV1 Yes Yes Yes No No No No
VP8 Yes Yes Yes No Yes Yes No
VP9 Yes Yes Yes No Yes Yes No
MVC No Yes No No No No No
M-JPEG No Windows Yes Yes Yes Yes No
JPEG 2000 No No Yes No No No Standard Yes
Apple ProRes No No No Yes No No Yes

 

See Appendix E for a list of file-type extensions which identify the container types.

Patent Rights & Intellectual Property Issues

Patent and royalty issues may determine your choice of container. Some licenses cover patented technologies which are provided free of charge while others levy a fee on all content created with them.

The MPEG standards are usually patent encumbered and require license fees to be paid. The Matroška format (MKV) is license free and whilst the WebM profile might use codecs built with patented technology, they are provided in a royalty free manner.

Format Licensing
Matroška Free and open
WebM Royalty free
Advanced Systems Format Container is free but codecs stored in it may require license fees.
Ogg Open-source
SMPTE Material Exchange Format Patent-free
MPEG program stream Patent-free
MPEG transport stream Patent-free
QuickTime File Format Proprietary
Audio Video Interleave Proprietary
Flash Video Proprietary
Real Media Variable Bitrate Proprietary
DivX Media Format Proprietary
MPEG-4 Parts 1,12,14 and 15 Patent encumbered
BDAV MPEG-2 transport stream Patent encumbered
Video Object Patent encumbered
Enhanced VOB Patent encumbered
3GPP Patent encumbered
3GPP2 Patent encumbered
Flash Video Patent encumbered

 


Note that there may be royalties on the video codecs but not on the container formats that carry them. Since they are standardised separately, any patent encumbrance could apply to either, both or neither.


Conclusion

Choose your container carefully to avoid limiting your choice of codecs.

For deployment on the Internet, the WebM and MP4 containers together offer the widest possible support. Implement your <video> tag in a web page so it will fall back to secondary formats if the first one is not supported. The browser will gracefully handle the choice of media. Note that the video and audio codecs supported by WebM are chosen because they are royalty free and you may require special tools to encode the content.

Production Video can be managed with MP4 files but the codecs must be profiled correctly to avoid losses. The AVC and HEVC codecs both support lossless profiles. Apple ProRes is popular for TV production work but can only be carried in QuickTime or MXF containers.

The MKV (Matroška), MP4 and MXF containers are the most versatile for building automated workflows.


The MXF file format lacks support for some end-user deployment codecs.


Part of a series supported by

You might also like...

The Big Guide To OTT - The Book

The Big Guide To OTT ‘The Book’ provides deep insights into the technology that is enabling a new media industry. The Book is a huge collection of technical reference content. It contains 31 articles (216 pages… 64,000 words!) that exhaustively explore the technology and…

An Introduction To Network Observability

The more complex and intricate IP networks and cloud infrastructures become, the greater the potential for unwelcome dynamics in the system, and the greater the need for rich, reliable, real-time data about performance and error rates.

2024 BEITC Update: ATSC 3.0 Broadcast Positioning Systems

Move over, WWV and GPS. New information about Broadcast Positioning Systems presented at BEITC 2024 provides insight into work on a crucial, common view OTA, highly precision, public time reference that ATSC 3.0 broadcasters can easily provide.

Next-Gen 5G Contribution: Part 2 - MEC & The Disruptive Potential Of 5G

The migration of the core network functionality of 5G to virtualized or cloud-native infrastructure opens up new capabilities like MEC which have the potential to disrupt current approaches to remote production contribution networks.

Designing IP Broadcast Systems: Addressing & Packet Delivery

How layer-3 and layer-2 addresses work together to deliver data link layer packets and frames across networks to improve efficiency and reduce congestion.