Modern software applications rely on libraries that are pre-compiled and provide specific functionality that is common across many different applications. For example, the “printf” statement often found in C/C++ that is used to print formatted strings, and the SSH library that is required to provide secure connectivity to a program. It is possible for a development team to re-write printf or SSH but what would be the point? The current versions are well established, have stood the test of time, are regularly tested for security vulnerabilities, and have strong maintenance teams supporting them.

When developing applications the software teams will often use multiple libraries from open-source or third-party vendors. When the application needs to print a message to a command line console, or log file, it may well use “printf” or one of its variants from the “stdio” library. A common interface is provided through the library’s header files and the precompiled code is available within the computer’s library repository. 

Generally, the most recent version of the library is available within the computers operating system installation, and everything works just fine. However, the developers may well require specific versions of the libraries to be used, and good development practice goes as far to stipulate this.

Being Secure

Software libraries drastically improve development times, and they have the bonus that they can provide protection against software vulnerabilities. CVE.org keeps an accurate record of all software library vulnerabilities that have been found and the status of the fixes that are available through software versioning. Software vendors keen to proactively maintain high levels of security will register with organizations such as CVE.org and receive bulletins when one of the libraries they are using has encountered a security vulnerability. When the secure version is released and available, the vendor will recompile their microservice and deploy it to their clients.

This simple example of deployment demonstrates how keeping track of where third-party libraries are installed within a server can be challenging, especially if the broadcast engineers maintaining the software have a thousand other things to do. However, microservices architectures have a solution to this through the concept of containers.

Container Images

Libraries, often referred to as dependencies form the core components that keep the software bug free. If an application calls the wrong version of a library, then it’s entirely possible that the software may be buggy or even crash unexpectedly. Containers are a method for creating images that not only contain the microservice application, but also the correct version of each dependency needed to make the software operate reliably. The container is a single file and can be deployed directly to the server giving the software a safe environment to execute from.

Keeping microservices and their corresponding containers functionally focused empowers the concept of continuous delivery (CD). In software engineering, CD is the work process that encourages regular deployment of code. Not only does this allow new features to be provided quickly, but it also drives a culture of regularly updating code. This is particularly useful for both new feature requests and bug fixes, as well as installing new software patches. Broadcasters should be in a position where they can employ CD to maintain the high levels of security that is needed when storing and transmitting high value media assets.

Automating Containers

In a typical broadcast infrastructure, there could be hundreds of different containers all providing different functionality, and each of these could have multiple instances, resulting in potentially thousands of containers. Managing this number of containers and all the associated hardware required to execute the programs would be virtually impossible if operated manually. Luckily, there is an automated solution to this through orchestration and deployment software, one version of this is call Kubernetes.

A level of hardware abstraction is provided by Kubernetes that treats the hardware as a cluster consisting of multiple servers and groups them together so that as a whole, they provide a combined resource of CPU power and RAM size. For example, if the cluster consisted of twenty-servers, all with a CPU measure of 1000 each and 128GB of RAM each, they would be combined to provide a CPU measure of 20,000 and RAM size of 2,560GB. From the perspective of the system admin or end user, we no longer need to think in terms of specific machines, but instead think in terms of the abstract of clusters, as the Kubernetes orchestrator takes care of the pod deployment within the cluster of nodes.

Abstracting Hardware Through Nodes

The lowest unit of resource within Kubernetes is the node, and this can be any physical type of machine, from laptop computers to high-end blade server. By referring to the hardware in terms of nodes, the underlying hardware can take on any guise, thus making the system both incredibly flexible and highly resilient. New servers (nodes) can be added to the cluster at any time to both improve resilience and hardware capacity.

Containers with their microservice programs can be run on any node within the cluster, and they can also move around different nodes depending on when the workflow demands them. This further adds to the concept of flexibility, scalability, and resilience, because as far as the Kubernetes orchestration and controller is concerned, the microservice and underlying hardware are separated (at least logically) from one-another. Therefore, if a node fails, or more nodes are added during peak demand, the microservice can be moved to a new node without human intervention. In other words, the scalability and resilience are fully automatic.

Rethinking Storage

This moveability of applications brings into question the idea of storage. When software is executed on a single machine, we know exactly where any files are stored as they will be on the local disk drive. This assumption cannot be made when we look at clusters and nodes as the application could be moved to a different server leaving the files on the old server. Clearly, this is not acceptable. To overcome this challenge, the cluster must have a persistent storage device attached to so that it is visible to all nodes. This may be a network storage device, or dedicated server that is assigned a network storage role.

One interesting aspect of the cluster architecture under Kubernetes is that the physical devices that make up the nodes and persistent storage do not need to be in the same geographical location. Different nodes could be in on- and off-prem datacenters, either privately within the broadcaster’s domain or scaled to the public cloud. This provides huge potential when considering flexibility as well as resilience.

Pod Application Execution Abstraction

Although the node is the smallest unit of hardware resource that makes up the cluster, the pod is the smallest unit of execution. By grouping containers together into a pod, related services can share the same resource and local network thus improving communications and reducing latency. The pods allow containers to run within them, and the pods are allocated to the nodes that form the cluster.

The concept of pods is particularly helpful when considering workflows that consist of several containers. For example, a transcoder may consist of a YUV to RGB converter container, an RGB to HEVC transcoder container, and an HLS streaming container. By configuring them all in the same pod, their messaging and signal exchange can be kept within either the same node, or physically close cluster of nodes so that the data throughput and processing is kept high, while latency is kept low.

Scaling & Resilience

Kubernetes adds another layer of management to the infrastructure as it detects broken or unresponsive nodes and will deploy the pods and containers associated with them to other nodes, thus providing automated resilience. And when the pods and containers are reaching the limits of their allocated node resource, Kubernetes will bring new nodes online to join the cluster using pre-configured rules.

These nodes can be within the datacenter or from an external supplier such as the public cloud.

When allocating external resource from public cloud providers and off-prem datacenters, the network design and architecture must also be taken into consideration as any congestion could result in high and unpredictable latency.

From this very brief introduction to how the components that make up microservices operate within containers, pods, clusters and nodes, simply taking a stand-alone application and attempting to just move it into a microservices environment is both futile and very difficult to achieve, if not impossible. To truly take advantage of all the benefits that microservices provide, the code must be written from the ground-up to deliver scalability, flexibility and resilience.