The previous two articles addressed the evolution of MPEG audio. Starting with MPEG-1 and MPEG-2 for layers 1 to 3. Later, MPEG-2 introduced AAC as a better but non-backwards compatible alternative in Part 7. MPEG-4 improves on the coding and MPEG-D adds a hybrid codec for speech and music.

MPEG-4 Part 3

Where earlier MPEG Audio standards concentrated on general audio coding, MPEG-4 covers a much wider range of target applications. It manages a collection of versatile codecs through a unified interface. The new codecs are sometimes a more efficient choice than AAC:

Building your own custom player for these special codecs might incur a patent licensing liability.

High Efficiency AAC Coding (HE-AAC/AAC+)

High Efficiency AAC (HE-AAC) refines the coding techniques but remains compatible with MPEG-2 Part 7. It can deliver almost CD-quality sound at 32 Kbps.

HE-AAC is known by a variety of other names:

These are some important features:

• Spectral Band Replication (SBR).
• Parametric Stereo (SSC).
• Perceptual Noise Substitution (PNS).
• Long Term Predictor (LTP).
• Low Delay (AAC-LD).
• MPEG-4 Scalable to Lossless (SLS).
• AAC Scalable Sample Rate (SSR).
• Structured Audio (SA).
• Text To Speech (TTSI).
• Error Resilience (ER).

Structured Audio and text to speech are extremely compact because they describe a sound that the player renders entirely in the receiving client. This can deliver performance as low as 100bps.

There are a many historical versions with profiles adding more variants:

Tools & Technologies

The coding tools were reorganized in MPEG-4 to make them more flexible. New tools were added and older tools were refactored into separate items. They are now described as Audio Objects, each one having a specific identity and purpose. Some of them are containers for descriptive information. This is an additional layer of abstraction facilitating profile definitions.

These are the basic audio objects and the technologies that are used inside them. Refer to the MPEG-4 Part 3 sub-parts for functional descriptions of these tools and how they are mapped into the profiles. The sub-part references are all in the MPEG-4 Part 3 standard:

Spectral Band Replication (SBR)

Spectral Band Replication discards redundant harmonic components in the encoder but reconstructs them by replicating the lower frequencies to derive suitable replacements in the player. This can be used with any codec.

A typical stream of audio might be coded to a target bitrate of around 128kbps. This would reproduce all frequencies up to 15kHz with a small reduction in the frequency response at the top end.

SBR cuts off the incoming frequencies at around 7.5kHz. This loses a lot of the detail but reduces the bitrate to 64kbps.

Then the higher band from 7.5kHz to 15kHz is processed through a more aggressive compression tool. This generates a description of the high frequency sounds that can be used in the decoder to reconstruct them from the lower order harmonics. The description is carried in auxiliary segments within the stream and only adds 1.5kbps to the bitstream (65.5 kbps in total).

The player transposes the lower frequencies into the upper band where it can filter and mix them in using the descriptions in the auxiliary segments.

Parametric Stereo (PS) & SinuSoidal Coding (SSC)

Parametric stereo exploits the similarity between the left and right channels to code them more efficiently.

The two channels are mixed down into a single monophonic channel and coded at full resolution. This is a base from which two differential channels can be derived. Those differences can be coded to a 3-kbps bitrate using SinuSoidal Coding.

The player decodes the mono channel and applies the differences to make the left and right outputs.

Instead of delivering two full bitrate channels, the encoder delivers 1 full bitrate channel and two very low bitrate differentials.

Perceptual Noise Substitution (PNS)

The bitrate gains from using PNS are often not worth the computational workload when the audio is of a high quality.

For noisy audio sources, the noise can be filtered out and described as control parameters for a pseudorandom noise generator in the player where they can be recreated.

Scalable Sample Rate (SSR)

Scaling the audio coding by splitting at the sample level is an interesting alternative to using base and enhancement layers.

If we de-interleave CD audio into 3 scalable streams then stream 1 carries the first sample, stream 2 the second and stream 3 the third and perhaps the fourth. The next sample is added to stream 1 and so on. This yields two 11 kHz sample streams and one 22 kHz stream which can be used by the target device in any combination.

A low-quality service can be reconstructed from one stream or all of them can be combined to reconstruct the original sample stream.

Error Resilience (ER)

Profiles & Audio Objects

A profile could use a single Audio Object while other profiles stack the tools hierarchically to make more efficient and sophisticated coders. Complexity requires more computational effort and increases the latency:

• MPEG-2 AAC-LC profile just uses the Low Complexity AAC-LC object.
• MPEG-4 AAC-LC adds Perceptual Noise Substitution (PNS).
• MPEG-4 HE-AAC v1 adds Spectral Band Replication (SBR).
• MPEG-4 HE-AAC v2 Adds Parametric Stereo (PS).

Because they are hierarchical, HE-AAC v2 players can decode any of the lower stacked levels.

These are the standardized profiles. Organizations such as Fraunhofer create their own proprietary profiles:

The Fraunhofer Scalable Lossless Coding (HD-AAC) is not the same as the SLS support defined by the MPEG-4 standard.

Refer to section 1.5 of the MPEG-4 Part 3 standard for a detailed description of the Audio Objects and how they are mapped to the profiles.

Additional MIME Type

Because HE-AAC is coded differently to classic AAC, a new MIME type is needed so that browsers can distinguish between the two formats:

Related Standards

The version date indicates the most recent base standard, corrigenda or amendment. Although the latest versions are indicated, earlier versions may contain relevant information that is removed from later standards. Some devices may be compatible only with an earlier version and you should use that when developing your services for them.

There is a gradual refactoring of the MPEG standards so they can benefit from reusing supporting technologies without needing to repeat them. The MPEG-D and Coding Independent Code Points standards are examples of that as are the ISO 23XXX group of MPEG related standards which provide additional infrastructural support outside of the individual coding specifications.

Patent Licenses

Patents for MPEG-4 Audio coding are managed by Via Licensing. Contact them for a license if you design and sell an Encoder or Decoder (Player) of your own.

Content owners do not need a license to distribute their MPEG Audio content.

Patents for AAC baseline technologies expire in 2028 and some newer extensions will have active patents until 2031.

Conclusion

Audio and video compression is a complex subject. We balance it here at a level sufficient to explain the fundamentals whilst avoiding a deep dive into the ‘Rabbit Hole’. Consult the MPEG-4 Part 3 standard if you need to explore MPEG Audio coding in greater detail.

The AAC audio standard is increasingly being used with High-Definition TV services (HDTV). This is supported by the DVB standards that are managed by ETSI. HE-AAC is particularly relevant for mobile TV using DVB-H.

Digital radio services such as DAB+ and Digital Radio Mondiale are also adopting HE-AAC

Our recent articles have traced the history of MPEG audio coding from MPEG-1 Layer 1 up to the latest hybrid USAC codec. We are not quite finished with Audio yet. ST 2110-3x and AES are on the horizon.

These Appendix articles contain additional information you may find useful: