Standards: Part 4 - Standards For Media Container Files
Once logged in you can set content preferences to let us know what you are interested in and whether you want to receive email from us.
Already Registered? Login
This article describes the various codecs in common use and their symbiotic relationship to the media container files which are essential when it comes to packaging the resulting content for storage or delivery.
This article is part of our growing series on Standards.
There is an overview of all 26 articles in Part 1 - An Introduction To Standards.
Media container files are as the name suggests, a means by which you can store your media in a container file for ease of production and deployment. They also enable the addition of time-synchronised text to support subtitles. The text can include mark-up to add styling during presentation. Special cases such as URL tracks use plain-text but are marked for special treatment in the player. Containers can also include descriptive metadata to facilitate the production.
What Do We Need To Carry?
Modern containers can carry multiple video streams and many audio tracks for language, accessibility support and surround sound. Other media types can be stored and synchronised with the video and audio for subtitles and interactive presentations.
| Content | Notes |
|---|---|
| Video | Lossless for production, lossy for deployment. |
| Audio | High quality for production but can be down-sampled for deployment. |
| Audio description | Additional audio track for accessibility. |
| Surround-sound | Multiple tracks of spatial audio. |
| Subtitles | Stored as Unicode text in one or more languages. |
| Metadata | Describes the format and semantics of the content. |
In due course, other kinds of data will be needed to support new formats like graphical point clouds and 3D spatial information for integration with the Metaverse.
Containers Vs. Codecs
Confusion arises because the containers sometimes have the same name as a codec stored inside them. The H.264 codec has several alternative names which adds to the uncertainty.
Currently , the most popular codecs for end-user deployment are:
- AV1
- AVC
- HEVC
- MPEG-2
- VP9
These can all be carried in the ISO standard MP4 container along with anything else that has been registered as a compatible media type. Virtually anything can be stored in a Matroška container file.
Sometimes you will see containers described as being lossless but the containers themselves do not affect the video quality. They are simply a means of packaging the output of a codec. It is the codec that might be lossy and not the container!
Base Media File Formats
Base Media File Formats are foundations that provide a basic structure for storing arbitrary data of any kind. Other container formats are derived from them either by extending them or by using profiles.
| Format | Description |
|---|---|
| ASF | The Advanced Systems Format (ASF) facilitates Windows Media content delivery. |
| MKV | The Matroška format (pronounced Matroshka) is used as the basis for WebM which only needs a sub-set of its functionality. It is named after the nested wooden dolls from Russia which illustrate the internal storage structure. The profiles dictate which codecs can be used. |
| RIFF | The Resource Interchange File Format is the basis for the AIFF files that carry extracted CD audio. |
| ISOBMFF | Described in MPEG-4 Part 12 and extended in MPEG-4 Part 15. |
The most versatile container formats are the ISO Base Media Format Files (ISOBMFF) also known as MP4 (MPEG) and the MKV (Matroška) format. These offer the widest range of codec compatibility and support across multiple platforms. The MP4 format in particular is well supported in all browsers and many other kinds of devices. The AVI file format is almost as versatile but lacks support for some modern codecs.
Popular Container Types
These are the most popular container types built from the base media formats:
| Format | Description |
|---|---|
| WebM | Web Media Files were designed by Google and are royalty free. They are optimised for use in HTML5 web pages and supported by the latest web browser versions. Based on a profile of the Matroška base media format, WebM only carries AV1, VP8 and VP9 video codecs. Audio is compressed with Opus or Vorbis codecs. |
| MKV | The Matroška base media format supports a variety of profiles for different applications. It is currently the only container format that supports Multi-View-Coding (MVC). This container could be very useful in an automated workflow. Although it is a foundational format, it can be used directly and bounded with profiles. |
| MP4 | MPEG-4 extends the ISO Base Media Format to support additional codecs. |
| QTFF | QuickTime files are usually stored in MOV files. Modern apps use the MPEG-4 file format instead. Consequently, QuickTime is gradually fading into history. It is important as a container for Apple ProRes coded video. |
| ASF | The Advanced System Format is useful for moving content around production workflows and is partially supported inside MXF containers. |
| AVI | The Audio Video Interleave files are a versatile container format based on the RIFF file structures. AVI is useful for archival purposes although it is becoming less popular as new container formats evolve to replace it. |
| MXF | SMPTE 377M describes the Material Exchange Format files used in production environments. This is the only alternative to QuickTime for carrying ProRes video content. The specification includes metadata support and some compatibility with Advanced Authoring Format (AAF) tools. |
Valuable archive footage may have been stored in a deprecated format. Keeping old hardware running with ancient OS versions allows you to run a conversion process to extract the legacy material for repurposing.
ISO MPEG-4 Containers
The ISO Base Media File Format is based on the same DNA as the Apple MOV file format developed for QuickTime. Apple designed this file structure to support multiple simultaneous media tracks of any type.
Combine MPEG-4 Parts 12, 14 and 15 and RFC 6381 to fully understand the MP4 container file structure. Then examine the MP4 registry to find descriptions of the internal chunk storage ATOMs.
| Standard | Description |
|---|---|
| MPEG-4 Part 1:2010 | MPEG-4 Part 1 describes the version 1 storage format. That is obsoleted by Part 14. The rest of the Part 1 standard is undergoing revision. |
| MPEG-4 Part 12:2022 | ISO Base Media File Format (ISOBMFF) is technically identical to the JPEG 2000 file storage defined in ISO 15444 Part 12 which it replaces. A new revision is under development. |
| MPEG-4 Part 14:2020 | MPEG-4 file format version 2 completely replaces the specification in Part 1 and is based on Part 12. |
| MPEG-4 Part 15:2022 | Describes how to package and store MPEG-4 Part 10 AVC and HEVC video. It is based on Part 12. Amendments have been published. |
| IETF RFC 6381 | Specifies how supported media types are described in the metadata. |
| MP4 ATOM registry |
The coded content is stored as structured chunks of data called ATOMs. A complete list of the ATOM types is available from the MP4 Registration Authority. (https://mp4ra.org/registered-types/boxes). |
Containers Vs. Compatible Codecs
Certain combinations of video and audio codecs are frequently used together. If you defy the conventions and use an unusual combination, your viewers may only see the video but not hear the sound, or vice-versa. The container type dictates your choice of audio and video codecs.
| Format | WebM | MKV | MP4 | QTFF | ASF | AVI | MXF |
|---|---|---|---|---|---|---|---|
| MPEG-1 Video | No | Yes | Yes | Yes | Yes | Yes | Yes |
| MPEG-2 Video | No | Yes | Yes | Yes | Yes | Yes | Yes |
| MPEG-4 Part 2 Visual | No | Yes | Yes | Partial | Yes | Yes | Yes |
| AVC (H.264) | No | Yes | Yes | Yes | Yes | Yes | Yes |
| HEVC (H.265) | No | Yes | Yes | Yes | Yes | Yes | Yes |
| AV1 | Yes | Yes | Yes | No | No | No | No |
| VP8 | Yes | Yes | Yes | No | Yes | Yes | No |
| VP9 | Yes | Yes | Yes | No | Yes | Yes | No |
| MVC | No | Yes | No | No | No | No | No |
| M-JPEG | No | Windows | Yes | Yes | Yes | Yes | No |
| JPEG 2000 | No | No | Yes | No | No | No Standard | Yes |
| Apple ProRes | No | No | No | Yes | No | No | Yes |
See Appendix E for a list of file-type extensions which identify the container types.
Patent Rights & Intellectual Property Issues
Patent and royalty issues may determine your choice of container. Some licenses cover patented technologies which are provided free of charge while others levy a fee on all content created with them.
The MPEG standards are usually patent encumbered and require license fees to be paid. The Matroška format (MKV) is license free and whilst the WebM profile might use codecs built with patented technology, they are provided in a royalty free manner.
| Format | Licensing |
|---|---|
| Matroška | Free and open |
| WebM | Royalty free |
| Advanced Systems Format | Container is free but codecs stored in it may require license fees. |
| Ogg | Open-source |
| SMPTE Material Exchange Format | Patent-free |
| MPEG program stream | Patent-free |
| MPEG transport stream | Patent-free |
| QuickTime File Format | Proprietary |
| Audio Video Interleave | Proprietary |
| Flash Video | Proprietary |
| Real Media Variable Bitrate | Proprietary |
| DivX Media Format | Proprietary |
| MPEG-4 Parts 1,12,14 and 15 | Patent encumbered |
| BDAV MPEG-2 transport stream | Patent encumbered |
| Video Object | Patent encumbered |
| Enhanced VOB | Patent encumbered |
| 3GPP | Patent encumbered |
| 3GPP2 | Patent encumbered |
| Flash Video | Patent encumbered |
Note that there may be royalties on the video codecs but not on the container formats that carry them. Since they are standardised separately, any patent encumbrance could apply to either, both or neither.
Conclusion
Choose your container carefully to avoid limiting your choice of codecs.
For deployment on the Internet, the WebM and MP4 containers together offer the widest possible support. Implement your <video> tag in a web page so it will fall back to secondary formats if the first one is not supported. The browser will gracefully handle the choice of media. Note that the video and audio codecs supported by WebM are chosen because they are royalty free and you may require special tools to encode the content.
Production Video can be managed with MP4 files but the codecs must be profiled correctly to avoid losses. The AVC and HEVC codecs both support lossless profiles. Apple ProRes is popular for TV production work but can only be carried in QuickTime or MXF containers.
The MKV (Matroška), MP4 and MXF containers are the most versatile for building automated workflows.
The MXF file format lacks support for some end-user deployment codecs.
These Appendix articles contain additional information you may find useful:
Part of a series supported by
You might also like...
Designing IP Broadcast Systems - The Book
Designing IP Broadcast Systems is another massive body of research driven work - with over 27,000 words in 18 articles, in a free 84 page eBook. It provides extensive insight into the technology and engineering methodology required to create practical IP based broadcast…
Demands On Production With HDR & WCG
The adoption of HDR requires adjustments in workflow that place different requirements on both people and technology, especially when multiple formats are required simultaneously.
If It Ain’t Broke Still Fix It: Part 2 - Security
The old broadcasting adage: ‘if it ain’t broke don’t fix it’ is no longer relevant and potentially highly dangerous, especially when we consider the security implications of not updating software and operating systems.
Standards: Part 21 - The MPEG, AES & Other Containers
Here we discuss how raw essence data needs to be serialized so it can be stored in media container files. We also describe the various media container file formats and their evolution.
NDI For Broadcast: Part 3 – Bridging The Gap
This third and for now, final part of our mini-series exploring NDI and its place in broadcast infrastructure moves on to a trio of tools released with NDI 5.0 which are all aimed at facilitating remote and collaborative workflows; NDI Audio,…