Metadata is vital to managing your collection of assets in an archive or librarian system. It describes relationships between different media assets. It is also vital when managing the production workflow.

During playback, metadata driven signals trigger remote actions to synchronously present supporting assets alongside the viewer in the client player.

It is the essential glue that steers the production processes and integrates all your media assets.

The ST 2110 Metadata Standards

SMPTE has already published or is working on several standards covering the use of metadata in an ST 2110 based IP network.

ST 2110-40 - Carriage Of SMPTE ST 291-1 Ancillary Data

Ancillary data (ANC) extracted from an SDI stream is delivered in real-time alongside the video and audio essence data. They all travel in separate RTP streams around the IP network. The system architecture is described in ST 2110-10 which covers timing and all the common characteristics of the different streams. The streams are all synchronized to a common reference clock.

The individual packets of ANC data are described in SMPTE standard ST 291 Part 1. The source of this data is the VANC and HANC data spaces within an SDI signal. Those data spaces are described in RP 291 Part 2 for high-definition TV services.

The code that supports this process is quite simple to build:

• The HANC and VANC ancillary data spaces are de-embedded from the incoming SDI feed.
• The individual data items are parsed out and stored in local variables.
• The metadata is formatted into ST 291-1 packets.
• The ST 291 packets are marshalled into an RTP stream for output.

ST 291 Part 1

ST 291-1 is a robust metadata system designed to carry anything that could potentially appear in VANC and HANC data spaces within an incoming SDI feed.

There are two similar kinds of Ancillary Data Packets described (Type 1 & 2). Type 1 is more useful when lengthy user data being transmitted (in excess of 255 words). Type 2 supports a wider variety of message types for delivering controls and signals.

The packets all carry a packet Description ID (DID) code that allows the receiving client player to handle each kind of packet correctly. Type 2 extends this with a Secondary ID (SDID) value.

High-definition ANC data formatting is described in recommended practice RP 291-2.

SMPTE often publishes recommended practice documents and sometimes promotes them to full standards when they are ready. The ST and RP documents therefore share a common numbering space.

Data ID & Secondary ID Codes

The Data ID (DID) and Secondary Data ID (SDID) values describe the type of ANC packet being transmitted. The Type 1 packets contain a single DID value while the Type 2 packets have A DID plus an additional SDID value that allows for a greater range of semantic meanings:

• Type 1 - DID + Sequence Number.
• Type 2 - DID + SDID.

The DID and SDID formats are set out in ST 291-1 and the range of possible values are maintained in a registry managed by SMPTE at this URL:

Note that a different URL is described at the top of Page 5 in the ST 291 standard but the target web page for it is missing.

Prior to the online registry, this data was published in the SMPTE RP 291 document. This is now out of date and has been withdrawn.

Refer to Section 4 in the ST 291-1 standard for a description of how the registry is operated and how to request new ID values.

Once a value has been registered against a DID or SDID value, the meaning cannot ever be altered nor can it be deleted from the registry. It might be deprecated though.

Type 1 ANC Packets

Type 1 packets can carry these payloads in the user data area:

• Audio Data.
• Camera position.
• Error Detection and Handling.
• Continuous sequences of related packets
• A generic Time Labelling format is under development.

Here is the internal structure for a Type 1 ANC packet:

These are the various parts of the packet:

You need not worry about this if there are no 8-bit legacy systems in your enterprise.

Type 2 ANC Packets

Type 2 packets can carry a wider variety of payloads:

• Acquisition Metadata Sets for Video Camera Parameters.
• AFD and Bar Data.
• Ancillary Time Code.
• ANSI/SCTE 104 messages.
• Compressed Audio Metadata.
• Data broadcast (DTV).
• DVB/SCTE VBI data.
• EIA 608 Data mapping.
• EIA 708B Data mapping.
• Extended HDR/WCG for SDI.
• Film Codes.
• HD-SDTI transport in active frame space.
• KLV Metadata transport.
• Link Encryption Messages & Metadata.
• Lip Sync data as specified by ST 2064-1.
• Metadata to monitor errors of audio and video signals on a broadcasting chain.
• MPEG recoding data.
• MPEG TS packets.
• Packing UMID and Program Identification Label Data into SMPTE 291M Ancillary Data Packets.
• Pan-Scan Data.
• Payload Identification.
• Program Description.
• SDTI transport in active frame space.
• Stereoscopic 3D Frame Compatible Packing and Signaling.
• Structure of inter-station control data conveyed by ancillary data packets.
• Subtitling Distribution packet (SDP).
• Time Code for High Frame Rate Signals.
• Transport of ANC packet in an ANC Multi-packet.
• Two Frame Marker.
• VBI Data.
• Vertical Ancillary Data Mapping of KLV Formatted HDR/WCG Metadata.
• WSS data per RDD 8.

Here is the internal structure for a type 2 ANC packet:

Type 2 packets replace the Data Block Number (DBN) with a Secondary Data ID (SDID) value. Hence, they cannot be used to construct sequences of related packets unless the sequencing is managed by the payload content.

Mapping ST 291 Packets Into RTP Streams

The mapping of ST 291 ANC data packets into the RTP streams is described in IETF RFC 8331. This is constrained by the requirements set out in ST 2110-10. It is important to obtain all of these standards first and cross-refer while you build, test and debug your IP network infrastructure.

RFC 8331 retains the location within the SDI ANC data spaces where the source metadata was found. It is described in terms of these properties:

• Line Number.
• Horizontal Offset.
• Stream Number.
• Interlaced Field (where applicable).

A stream of properly constructed RFC 8331 packets will allow the original SDI signal to be reconstructed for output at the destination.

Timestamps are based on the video fields and should be the same for all ANC packets derived from a single interlaced-field or progressive-frame.

If there is no ANC data to be transmitted, a ‘keep-alive’ packet is delivered as a heartbeat to maintain the connection.

Timing is critical to maintain synchronization. Keeping latency to a minimum is important too. Section 6 of the ST 2110-40 standard explores this in some depth.

The Session Description Protocol is widely used within the ST 2110 environment. This is also described in RFC 8331 and ST 2110-10. Refer to Section 7 of ST 2110-40 for details of how SDP is employed by metadata streams.

Section 7 also mentions that the flow identification semantics described in RFC 8331 should not be used as they are incompatible with ST 2110.

A format Specific Parameter is also described in section 7 and must be defined correctly to include the frame-rate.

ST 2110-41 - Fast Metadata Framework (FMX)

The Fast Metadata Framework (FMX) describes a more flexible format for transmitting metadata payloads in RTP packets. It is designed to use network resources more efficiently. ST 2110-40 can potentially consume many IP addresses. These are the design goals:

• Rapid delivery of metadata.
• Time aware and synchronized with the video and audio essence.
• Extensible architecture.
• Payload agnostic and can transport anything.
• Will use ST 2110-10 and ST 2059 to synchronize.
• Upwards compatible and will support ST 291 SDI ANC data.
• Useable with any kind of media stream.
• Can also run independently of any streams.
• Must require no changes to any other ST 2110 specifications.

FMX supports the delivery of time synchronized metadata associated with audio and video data streams or more loosely coupled metadata that does not need to be synchronous.

Like many other ST 2110 standards, this also describes RTP payloads and SDP messaging formats.

Section 5 describes the RTP packet format comprising two parts:

• RTP Header.
• RTP Payload.

The header is described in RFC 3550. ST 2110-41 adds constraints to profile this and render the packets compatible with the rest of ST 2110.

This is much simpler than ST 2110-40 in the way that it describes the format of the payload content. The RTP packet size is still constrained by the requirements set out in ST 2110-10.

The payload is constructed from a collection of Data Item Packages. An empty RTP packet can be delivered without any Data Item Packages for keep-alive purposes.

The Data Item Packages must not be split across multiple RTP packets. Their size must be a multiple of 32-bits but can vary in size. Any necessary padding is considered to be part of the Data Item Package and would be specified in the application document.

An alternative segmentation scheme for supporting an object like structure is described in Annex A.

Section 6 describes the SDP signaling message format which must conform to RFC 4566 and ST 2110-10. This mandates that the SSN value should describe an ST 2110-41 stream and enumerate any Data Item Types that may appear in the stream. Here is an example SDP message:

ST 2110-42 - FMX Payload For 2110 Technical Metadata

The original intent for this standard was to describe an object-based format for carrying the technical metadata about the video/audio streams. This is not intended to replace NMOS IS-04 or the SDP messaging protocols. The metadata payloads would be packaged as described in ST 2110-41. These are some of the items that were proposed:

Initial work was started in 2020 but the only public mention of ST 2110-42 is in the SMPTE quarterly reports. The most recent information suggests this standard may not be necessary as the requirements will be covered by other standards that are nearing completion. In particular ST 2127-2 is mentioned. As of February 2023, any further progress on ST 2110-42 appears to have halted for the time being.

ST 2127-2 - Mapping MGA Audio Metadata To ST 2110-41

ST 2127 Part 2 describes the mapping of Metadata-Guided Audio (MGA) Audio payloads into the Fast Metadata Framework (FMX). MGA is described more fully in ST 2127 Part 1 where it is used with MXF files and the FMX framework is described in ST 2110-41.

This standard was described in one of the SMPTE quarterly reports in 2022 as a partial replacement for ST 2110-42.

The standard is quite short and makes reference to the audio essence being carried by an ST 2110-30 stream controlled by the ST 2127-2 metadata carried in a ST 2110-41 framework using rules defined in ST 2127-1.

Acquire and read all four documents to understand the entire structure.

About ST 2110-43

The ST 2110-43 standard describes Timed Text Markup Language for Captions and Subtitles. Timed Text is essence media. It is only mentioned here because it shares many physical characteristics with metadata formats for delivery and is grouped with the ST 2110 metadata standards.

Related Standards

Refer to these other standards for supporting material that will assist in understanding and deploying ST 2110-4X metadata content:

Conclusion

Metadata management within the ST 2110 architecture is a work in progress. The ST 2110-40 standard is designed around the ancillary data found in an SDI signal.

A more generalized approach is specified in ST 2110-41 but this only describes the framework for carriage of the metadata. ST 2110-42 was intended to describe the message payloads and metadata formats but this work is currently suspended. Some audio metadata is described in ST 2127-2 and ST 2109 but there is currently no corresponding standardized description for video related metadata delivered via FMX.

For now, we must watch and wait.

These Appendix articles contain additional information you may find useful: