High-speed data and capacity requirements are now no longer exclusive to broadcasting, the proliferation of web sites and phone-apps has driven IT speed, capacity, reliability, and resilience way beyond the levels demanded by broadcasters. Big-data processing has further driven innovation and data storage systems capable of processing and streaming real-time video and audio is easily available using high-end IT solutions.

SDI broadcast infrastructures are highly tuned and optimized facilities that are difficult and expensive to maintain. Years of bolted-on tweaks and changes to facilitate new workflows add to the risk of outage and loss of transmission. Engineers are constantly chasing their tails trying to keep up with the changes to the system while maintaining high levels of reliability.

Web-site and phone-app service providers also encounter similar challenges to broadcasters, especially when trying to optimize data workflows, balance workloads, fine tune systems, and reduce the need for their team to constantly firefight.

IT Innovators Solutions

But IT innovators have been working hard to deliver high levels of intelligent automation and provide highly optimized infrastructures and workflows.

Although the first experimental AI systems go back to the 1940’s, recent advances in virtualized computation, networking, and storage has witnessed an explosion in the number of AI systems now available.

But what makes AI so important? Seemingly, we just analyze data collected from sensors and data-sets to help understand systems and provide a level of prediction. This method of statistical analysis has been used by engineers and scientists for hundreds of years. 

By looking at the number of users logging onto a web-site, the designers can set a pre-programmed service to spin up new virtualized servers to meet the increased demand. If analysis indicates the number of users double for four hours on a Friday at 6:00pm, pre-emptive action can be taken to spin up the servers in anticipation of the increased demand.

Systems are Dynamic

For as long as businesses have been able to harness information, managers have analyzed data to help predict the future using ever increasingly complex statistical methods. But this form of analysis assumes a static or very slow-moving system.

Statisticians and Business Intelligence Analysts spend a great deal of time trying to find correlation between different data sets. In the example above, analysis indicated web-site users doubled at 6:00pm on a Friday. Although this is a simple example, it’s difficult to establish with much certainty whether this is a trend or an anomaly. The data used for analysis is out of date before the information can be adequately scrutinized.

Context Aware

To improve the prediction, more data-sets would be needed so better correlations can be established. The prediction can never be 100% accurate, but accuracy is directly proportional to the number of correlated data-sets that can be established. AI comes into its own when we look at context-aware dynamic and fast-changing systems.

AI is a generic term that covers four different disciplines; reasoning, natural language processing, planning, and machine learning. Split into four more categories, machine learning includes supervised learning, unsupervised learning, reinforcement learning, and deep learning (or neural learning).

Traditional computer programming requires a set of logical rules to be provided for the data to be analyzed. Usually providing a series of decision trees, the programmer will parse the data and then provide binary outcomes. For example, if it’s 6:00pm on a Friday, then spin up two more virtual web-servers.

Programming is Too Slow

Changing the data to variables, such as “6:00pm” and “Friday”, allows generic rules to be established so that the time and day can be referenced from a database and easily changed. However, this method soon becomes complex and difficult to administer when the rules need to be adapted. 

It might become clear that Tuesdays also need an extra server at 6:00pm. This would require the programmer to add an addition to the rule or add a completely new rule. The example is trivial, but complexity of the program soon escalates, and efficient software maintenance is further compromised.

The method is slow and potentially problematic as the rules must be constantly adjusted to improve the efficiency of the system.

AI Generates Rules

Instead of manually programming rules for the data, AI takes the opposite approach and the data is used to automatically generate its own rules within the context of the data it’s working with in real-time. And that is the true power of AI.

Algorithms continuously monitor historic as well as real-time data and find accurate correlation between the data sets to help improve the future predictions for the system. This approach further improves efficiencies to take advantage of dynamic systems to constantly balance resource.

In the previous article in this series we looked at how advanced IT storage systems balance the user experience against the different storage resource available. An edit system would require instant low latency access to their edit masters and rushes, and this would be provided by the Solid-State-Device (SSD) and memory storage during the edit session.

Improving Efficiency

It would be uneconomical to continue to keep the media files in the high-speed storage when the edit session had finished so the files would need to be copied back to spinning disks. Although manual rules could be established to copy the files between the two storage mediums, AI provides a far more efficient method of providing this facility.

Algorithms can establish which files to copy based on previous sessions and the editor’s behavior. If the editor was previewing off-line rushes earlier in the day, the AI algorithm would associate these files with them and automatically copy the media files to SSD and memory storage associated with the edit suite for their edit session.

Real-Time Analysis

AI not only provides ever increasing efficient levels of automation, it does it without any human analysis, programming, or intervention. The algorithms learn the behavior of the storage dynamics and accurately predict which files need to be transferred between the different storage medium to improve the user experience and make the most of the resource available.

All this happens in near real-time making the traditional record-analyze-program methods now obsolete. In the next article in this series, we look at how AI has influenced and greatly improved the reliability and resilience of storage.