Managing Content Across Space and Time
The NetApp SG5600 StorageGRID Webscale Appliance offers high-density, cost-efficient object storage in an easy-to-deploy appliance.
If you handle media assets, you know how to store lots of data and manage its constant growth. You’ve put the right storage with the right performance into the right places. You’ve got asset management solutions that serve your workflows well. You have distribution tools that let you send data across the globe. The problem is that none of it works well together.
Examining the Problem
How many asset management systems do you have in your media operation? Three? Five? 10? Can any one of those MAMs display content across your entire operation? How much of your content shows up in more than one place? Those additional copies of data are driving more and more storage capacity without adding any real value to your business.
It’s likely that you’ve put your most active content on a high-performance storage tier, but how long should it stay there? Do your asset management tools automatically move content to another tier of lower-performing, less costly storage — such as disk, tape, or cloud — when the time is right? Are you using third-party software to interface with those archival tiers? How often do you check to make sure that your content hasn’t become corrupted (and thus useless) over the years?
What if you want to replace an asset management system or combine two systems into one?
Object Storage: One Common Source for All Content
Object storage addresses all of those issues and more. (An object is a file and its associated metadata.) First used in healthcare and long-term digital-document archives almost a decade ago, object storage has become popular thanks to adoption by cloud service providers and social media sites. Meanwhile the Amazon S3 interface is becoming a de facto standard, which has given MAM and other workflow-software vendors the incentive to interface to object storage.
Object storage can be integrated cloud storage. Vendors can use the same Amazon S3 interface for cloud storage as they do for object storage.
Now, early-adopter broadcasters have recognized the value of object storage. They are asking vendors to integrate with cloud storage, which, in effect, is a request to integrate with object-storage technology. Whether the cloud-storage business model is right for broadcasters or not, software vendors can use the same Amazon S3 interface for cloud storage as they do for object storage (the same technology as cloud storage, but deployed where you want it).
While early adopters are excited about cloud storage, they have concerns about security, governance, bandwidth, and access costs. Whether in the cloud or on-site, media operations are drawn to a storage platform that lets them control the three most difficult data-related problems they face today:
- Data management
- Data migration
- Data mobility
Therein lies the strength of object storage.
In an infrastructure based on object storage, you locate a piece of content simply by pulling it from a single object store — but having one object store doesn’t mean you have one particular storage array in one place. We all know that storage is made up of different platforms, possibly across several different geographies, with different levels of storage performance for different workflows in different parts of your operation. Object storage doesn’t change that; it simply allows it to look like a single repository to the outside world.
If you need multiple copies of content, the object store knows where they are. If a particular copy is no longer needed, object storage can remove it. With object stores, you set policies for your content so that copies and deletions are automatic. This automation is critical because when you have millions of files, if deletions don’t happen automatically, then they typically don’t happen at all, and your storage gets fuller and fuller.
Object storage lets each asset management system in your operation support the various workflows it does best rather than managing the file systems of many different storage platforms. Unique asset identifiers make it simple for your asset management systems to present the right content to the users who need it.
To ensure that your data is always available, some object stores have fault-tolerant architectures for uninterrupted operations, including during upgrades and infrastructure refreshes.
To keep your data intact over time, some object stores use digital fingerprinting, tamper detection, integrity verification, and automatic regeneration of suspect objects — not just on ingest, retrieval, replication, and migration, but also with objects that have gone unused for years.
Your content can gain or lose value over time. High-value content needs high-performance storage to keep it readily available, while low-value content can sit on lower-performance storage such as tape or cloud. An object store that can archive to tape and cloud using its native policy engine will automatically move content to the appropriate storage tier, thereby ensuring the right performance-to-cost ratio for that content’s value:
- Storage for active archives
- Tape or cloud storage for near-line access to less valued content
- Tape storage for long-term preservation
Collaboration, localized delivery, and cultural conformity often require you to store the same asset in more than one place. Some object stores can copy or move content automatically — either upon ingest or sometime later — based on the policies you set and triggered by metadata. As a result, you no longer have to spend hours copying every asset that is tagged in a certain way. Instead you can simply create a new policy associated with that tag, and the object store will automatically copy those objects to a second location.
For example, if yours is a news organization with footage of a natural disaster, you probably want every one of your news bureaus worldwide to have a copy of that content. The right object store can make it happen automatically, tracking where each copy goes and managing it when it gets there. Then, when that content is no longer relevant to the international bureaus, the same object store can delete all of the copies.
In the diagram above, content is stored based on user-driven policies triggered by metadata. Users can create a new policy associated with a tag, and the object store will automatically copy those objects to a second location.
With all of these advantages, object storage can organize your storage infrastructure in a way no other storage technology can. Using policies to manage objects across data tiers, locations, resiliency schemes, and retention periods, object stores manage your media across space and time.
Jason Danielson, Media and Entertainment Solutions, Product and Solution Marketing, NetApp
You might also like...
Designing IP Broadcast Systems
Designing IP Broadcast Systems is another massive body of research driven work - with over 27,000 words in 18 articles, in a free 84 page eBook. It provides extensive insight into the technology and engineering methodology required to create practical IP based broadcast…
Standards: Part 21 - The MPEG, AES & Other Containers
Here we discuss how raw essence data needs to be serialized so it can be stored in media container files. We also describe the various media container file formats and their evolution.
NDI For Broadcast: Part 3 – Bridging The Gap
This third and for now, final part of our mini-series exploring NDI and its place in broadcast infrastructure moves on to a trio of tools released with NDI 5.0 which are all aimed at facilitating remote and collaborative workflows; NDI Audio,…
Microphones: Part 2 - Design Principles
Successful microphones have been built working on a number of different principles. Those ideas will be looked at here.
Expanding Display Capabilities And The Quest For HDR & WCG
Broadcast image production is intrinsically linked to consumer displays and their capacity to reproduce High Dynamic Range and a Wide Color Gamut.