Scalable Dynamic Software For Broadcasters: Part 1 - Software Infrastructures
IP has succeeded in abstracting away the media essence from the underlying transport stream, and in doing so is providing scalable and dynamic solutions that are facilitated through cloud technologies and software.
All 12 articles in this series are now available in our free 88 page eBook ‘Scalable Dynamic Software For Broadcasters’ – download it HERE.
All articles are also available individually:
Although broadcasters have been using software for many years to process video and audio, it is only since the adoption of IP that software has been elevated to form a core component of the broadcaster’s infrastructure. Software defined services and networks all combine to deliver truly scalable and dynamic systems that can respond quickly and efficiently to the needs of the broadcaster’s business.
Software engineering has progressed massively in recent years and understanding modern delivery methods is equally important to understanding how systems operate. Gone are the huge monolithic software packages and incoming are the microservice apps through containerization to deliver highly efficient and scalable systems.
Importance Of Scalable Software
One of the major benefits of IP infrastructures is that it has expanded the type of technology that is now available for broadcasters. Hardware equipment still prevails but software COTS systems are playing an increasingly important role, especially when we look at scalable facilities.
Television, by its nature is very peaky in demand. Although playout systems are generally static in their resource requirements, program making is usually the opposite. But even playout is subject to variations in demand through pop-up and on-demand channels. Sitcom and light entertainment usually require studio capacity during the day or evening, leaving large amounts of hugely expensive equipment to lie dormant for hours or days on end. IP not only delivers a new flexible type of transport stream, but also delivers a new concept of flexibility as multiple video and audio services can be transported on a single connection to many areas within a facility.
The idea of multiple services on a single fiber isn’t new as frequency and time division multiplexing has facilitated this for a long time, but what is new is the ability to switch services throughout an infrastructure with off-the-shelf readily available resource such as ethernet switches and IP routers. SDI circuits facilitate point-to-point connectivity, but their huge cabling requirements and custom design interfaces limits their flexibility. By using high-capacity IP infrastructures, multiple video and audio services can be sent to and from any device within the network. The combination of routers, switches, and fiber cabling, empowers a holistic view of the network so that flexibility and scalability is built into the infrastructure from the beginning.
Moving To Packet Switching
Embedded in every IP packet header is the source and destination IP addresses of the device it originates from and the device it is destined to be delivered to. IT networks are auto-routing, that is the ethernet switch or IP router has a routing table built into it that knows which port to forward the packet to. Through protocols such as ECMP (Equal Cost Multi-Path), the look-up tables are updated automatically, and the packets arrive at their destination without human intervention. However, this method is prone to variable and unpredictable latency as the switches and routers have little knowledge (if any) of the network outside of their direct links. The introduction of SDNs (Software Defined Networks) has to a large extent solved these challenges.
SDNs divide the control and data planes within the network so that management software can be used to configure networks to route packets between devices. But more importantly, SDN provides a higher-level view of the whole network and forms part of the management layer. This delivers a single point of contact for the network management, so users have a high-level view of how a network is connected and configured. Although SDN solves routing challenges and helps make networks latency determinate, it leads to another challenge, and that is registration.
Device Registration
Traditional broadcast SDI and AES networks have registration built into them at the point of design through fixed cabling and labelling. However, this also builds in a lack of flexibility as SDI and AES routers often have their labels entered into a relatively static database, and their cabling is hard wired. This method has worked well for the best part of fifty years when the first centralized routers were built into TV stations but is now demonstrating a significant lack of flexibility that is proving costly for broadcasters.
IP offers another alternative but requires a level of abstraction through device registration. When an IP device is first connected to a network, its IP address must be either dynamically allocated at the point of connection or physically allocated to form a static IP address list. IT uses both these methods depending on the type of equipment being registered on the network. For example, a NAS storage device or gateway router will have a fixed IP address, but an ad-hoc user entering the WiFi zone will have their device’s IP address generated by the DHCP (Dynamic Host Control Protocol) server.
It is not uncommon for broadcasters to use static IP addresses for their devices including cameras, microphones, and production switchers etc. But this puts a massive administrative burden on the broadcaster as somebody somewhere must maintain a spreadsheet with all the device IP addresses, their designation, and configuration. A camera may have multiple IP addresses for UHD, HD, and SD video, or return audio and video paths, and each of these must have an associated SDP (Session Description Protocol) file describing the format of the video and audio, such as the frame rate, color space, or sampling rate etc., and each of these datapoints must be administered and managed. This results in a massive system and management overhead that could easily lead to mistakes.
Figure 1 – The monolithic software on the left encapsulates all three functions – Discovery, Architecture, and Registration. Changes to any of these functions requires a complete recompile and upgrade to the whole software thus increasing risk. Also, resilience and scalability are very difficult to provide as the various functional components cannot be easily separated. The microservices software on the right isolates the functions from each other so that they can be upgraded and deployed individually. Furthermore, resilience and scalability are much easier to achieve as a microservice can be installed on its own hardware or on shared hardware.
A better way of achieving this is through an automated form of device registration and configuration. NMOS has made some inroads into solving this, but proprietary systems have further expanded its capabilities to deliver device registration that not only allows all the sources and destinations to be available to other devices on the network, but also includes security validation to both protect the network and the high-value media assets traversing it.
From an operational perspective, it’s incredibly important to have the sources and destinations, along with their associated device configurations, instantly available to the equipment user. For example, a sound engineer operating a console for a live sports event will want to see a list of every available audio device on the network, along with their configurations, on a screen in front of them. In this context, referring to a spreadsheet that is probably out of date before it is printed, is not conducive to a reliable production.
Software Screwdrivers
Broadcasters can learn much from IT and one of these areas is logging and maintenance using well established software tools. These include many open-source software programs such as iPerf to test link connectivity, and Wireshark to record and analyze network stream grabs.
When we start to understand the IT mindset then an abundance of tools soon manifests themselves that don’t necessarily need custom hardware to run on. Instead, they use existing network servers and storage to allow us to look under the hood and find out what is going on in our networks, to a much greater level of detail and accuracy than we could ever achieve with SDI.
Python is a relatively new software scripting language that can facilitate the demands of the novice developer right up to the expert. With a few lines of code, we can generate scripts that can run automated batch sessions or trigger the recording of network recorders such as Wireshark when an unexpected event occurs. Furthermore, the availability of RESTFul APIs used with microservices makes workflow design much more flexible as the broadcaster can either write their own code to establish functional connectivity, or use the solutions supplied by their preferred partner vendor.
With the newfound freedom established with software tools broadcasters must look to adopt modern methods of software design and maintenance. Agile is prevalent in the coding community and is used as a software repository system, a collaboration tool, and a method for managing software releases, both in terms of functionality and people management to deliver software for the needs of the business.
Moving Forward
We’ve passed phase-1 of our migration into IP infrastructures. This was focused on moving low-latency and high-datarate video and audio reliably throughout an IP network. Now we’ve moved into phase-2 of our IP journey, that is automated plug-and-play integration and long-term flexibility and scalability.
Although there are several solutions to facilitating plug-and-play, of equal importance is the underlying infrastructure that supports it. We could run a device registration system on a single server, but this only provides part of the solution, as it doesn’t take into consideration the reliability and flexibility of the underlying infrastructure. Instead, a much more resilient approach is needed and microservices have gone a long way to deliver this.
Microservices are continuing to gather momentum as they deliver the security, resilience, and flexibility that broadcasters demand. They are self-contained and easy to manage, and deliver unprecedented levels of reliability. The next phase of our IP journey will require an understanding of how and why we use them. Article 2 in this series explains microservices.
Part of a series supported by
You might also like...
Designing IP Broadcast Systems - The Book
Designing IP Broadcast Systems is another massive body of research driven work - with over 27,000 words in 18 articles, in a free 84 page eBook. It provides extensive insight into the technology and engineering methodology required to create practical IP based broadcast…
Demands On Production With HDR & WCG
The adoption of HDR requires adjustments in workflow that place different requirements on both people and technology, especially when multiple formats are required simultaneously.
If It Ain’t Broke Still Fix It: Part 2 - Security
The old broadcasting adage: ‘if it ain’t broke don’t fix it’ is no longer relevant and potentially highly dangerous, especially when we consider the security implications of not updating software and operating systems.
Standards: Part 21 - The MPEG, AES & Other Containers
Here we discuss how raw essence data needs to be serialized so it can be stored in media container files. We also describe the various media container file formats and their evolution.
NDI For Broadcast: Part 3 – Bridging The Gap
This third and for now, final part of our mini-series exploring NDI and its place in broadcast infrastructure moves on to a trio of tools released with NDI 5.0 which are all aimed at facilitating remote and collaborative workflows; NDI Audio,…