Building Out Storage Infrastructure for 4K
A proper 4K production system demands not only greater storage capacity but also greater storage performance.
The media and entertainment industry is moving toward 4K and UHD because of the beautiful resolution and increased depth and color these formats offer. With more information in every frame, editors and other artists are empowered to be more creative — and they are better able to use the latest technology and tools to manipulate media files and create richer content that compels audiences to consume it.
As content creators move toward higher resolutions and frame rates, they put a great deal of stress on their existing infrastructures. Given the speed of this shift, some such companies already store hundreds of terabytes or even petabytes of media content. Far greater storage capacity is a clear and immediate must, but it is not the only important characteristic to consider in adapting storage infrastructure to work with today’s much larger file sizes.
Increasing only storage capacity to address 4K workflows is akin to using a screwdriver to hit a nail; it may yield some results, but it is not really the right tool for the job. In the case of adapting storage infrastructure to work with larger media files and formats, media facilities also must evaluate their requirements with respect to throughput, latency and input/output operations per second (IOPS). The ideal balance of these factors will be dictated by the unique aspects of an operation’s workflow and all integral processes.
Understanding 4K-Specific Challenges
Higher-resolution formats demand not only greater storage capacity but also greater storage performance. Just how much speed will depend on the type of encoding used. For example, while the 4K digital cinema files typically used for theatrical releases (and trailers and commercials displayed in theaters) are full-resolution 4096x2160, the 4K UHD files used on the broadcast side are 3840x2160 — smaller, but still about four times the resolution of 1080p. In either case, these high resolutions require a great deal of bandwidth, particularly when multiple users are performing diverse tasks such as color grading, editing and finishing.
4K images can range from 4K UHD broadcast files, which are 3840x2160 to full-resolution 4096x2160 files for theatrical releases. Both are about 4X the resolution of 1080p HD.
Not all 4K formats are equal, and the variety of 4K codecs on the market represents other key considerations such as which codecs will be used within the workflow, where, when and for which content? Some companies may decide to work with low-resolution files, also known as proxy files, during the editing process and do a final conform before delivery, and some others may choose to work at full resolution the whole time from ingest through edit to delivery.
All of these factors can make it difficult to design the ideal storage infrastructure, and consultation with a systems architect or solution designer can be of immense help in understanding how storage can best accommodate every step of the process, and where investment will bring the greatest benefit.
Examining Storage Requirements
Storage infrastructure has been and always will be the backbone of media workflow operations, and for them to be successful, this infrastructure must be reliable and sufficiently robust to address the requirements of the end-to-end workflow it supports. At the same time, the bottom line demands that content be stored in the most cost-effective manner possible without compromising the productivity of creatives and monetization potential of stored media.
Some companies need only massive “cold” archives in which to hold high-resolution content that is used infrequently. Others require large-scale “hot” archives with high bandwidth and speed, as well as low latency, capable of facilitating the frequent streaming of content. In other words, the specific mix of access, performance and cost requirements will differ for every media company. Each must first understand its various workflow requirements before it can build out storage infrastructure suitable for 4K content.
When selecting a 4K storage platform, the storage technology should be based on the characteristics of the workflow.
For those media facilities maintaining completely sequential workflows, such as typical editorial workflow, the traditional spinning disk hard drive delivers very good performance and appealing density at a very good price point. In case of media storage for random workflows, such as visual effects (vfx) where most of the operations are frame-based, the industry is seeing a greater embrace of SSD drives. Though these drives already are well-established in Online Transactional Processing (OLTP) workflows, today a changed market and far better economies of scale have brought SSD prices down, making them attractive for a broader range of applications.
Thus, in the media and entertainment industry, companies now are looking to leverage this technology to increase the speed of their workflows and performance of storage. However, using full SSD storage arrays to drive up performance for both sequential and random workflows is a less than ideal approach because most facilities would never need or require SSD-level performance across all of their operations. As a result, this is very expensive and, most of the time, completely underutilized. Naturally, media technology suppliers need to provide more cost-efficient options to customers.
Identifying the Optimal Mix of Storage Technologies
In the realm of 4K, the right storage technology would depend on the characteristics of the workflow. In some cases nearline SAS 7.2K HDDs or SAS 10K, 15K HDDs will get the job done. Or, maybe a combination of SSDs and HDDs using a well-established tiering technology to maintain multiple sets of storage in tandem and within a single architecture might offer a path towards better performance — higher throughput, lower latency and higher IOPS — at a lower cost.
This latter concept is quickly growing in popularity, largely because many media workflows have evolved to include both sequential and random elements. As a result, emerging hybrid architectures allow facilities to take advantage of different sets of storage technologies within a single infrastructure, with asset management systems steering content to the most appropriate storage tier. When powerful and sophisticated tiering technology is employed, this model provides the speed of SSD and the density of nearline SAS HDDs in a single platform with a seamless integration.
The use of hybrid architectures opens the door to much greater flexibility and efficiency in deploying and using storage resources to support 4K workflows. For many media companies, this model makes it far easier to effectively accommodate new file formats and/or codecs, increase the number of users and applications accessing content (independently and simultaneously), and be flexible with the processes being applied to stored media to create new content. With a storage infrastructure that addresses all of these elements, media facilities can move forward confidently in taking on 4K business and capitalizing on rapidly rising demand for higher-resolution deliverables.
Sergio Soto, Quantum Area Director Technical Sales for SW & LATAM
You might also like...
Designing IP Broadcast Systems - The Book
Designing IP Broadcast Systems is another massive body of research driven work - with over 27,000 words in 18 articles, in a free 84 page eBook. It provides extensive insight into the technology and engineering methodology required to create practical IP based broadcast…
Demands On Production With HDR & WCG
The adoption of HDR requires adjustments in workflow that place different requirements on both people and technology, especially when multiple formats are required simultaneously.
If It Ain’t Broke Still Fix It: Part 2 - Security
The old broadcasting adage: ‘if it ain’t broke don’t fix it’ is no longer relevant and potentially highly dangerous, especially when we consider the security implications of not updating software and operating systems.
Standards: Part 21 - The MPEG, AES & Other Containers
Here we discuss how raw essence data needs to be serialized so it can be stored in media container files. We also describe the various media container file formats and their evolution.
NDI For Broadcast: Part 3 – Bridging The Gap
This third and for now, final part of our mini-series exploring NDI and its place in broadcast infrastructure moves on to a trio of tools released with NDI 5.0 which are all aimed at facilitating remote and collaborative workflows; NDI Audio,…