Managing Content Across Space and Time

If you handle media assets, you know how to store lots of data and manage its constant growth. You’ve put the right storage with the right performance into the right places. You’ve got asset management solutions that serve your workflows well. You have distribution tools that let you send data across the globe. The problem is that none of it works well together.

Examining the Problem

How many asset management systems do you have in your media operation? Three? Five? 10? Can any one of those MAMs display content across your entire operation? How much of your content shows up in more than one place? Those additional copies of data are driving more and more storage capacity without adding any real value to your business.

It’s likely that you’ve put your most active content on a high-performance storage tier, but how long should it stay there? Do your asset management tools automatically move content to another tier of lower-performing, less costly storage — such as disk, tape, or cloud — when the time is right? Are you using third-party software to interface with those archival tiers? How often do you check to make sure that your content hasn’t become corrupted (and thus useless) over the years?

What if you want to replace an asset management system or combine two systems into one?

Object Storage: One Common Source for All Content

Object storage addresses all of those issues and more. (An object is a file and its associated metadata.) First used in healthcare and long-term digital-document archives almost a decade ago, object storage has become popular thanks to adoption by cloud service providers and social media sites. Meanwhile the Amazon S3 interface is becoming a de facto standard, which has given MAM and other workflow-software vendors the incentive to interface to object storage. 

Object storage can be integrated cloud storage. Vendors can use the same Amazon S3 interface for cloud storage as they do for object storage.

Object storage can be integrated cloud storage. Vendors can use the same Amazon S3 interface for cloud storage as they do for object storage.

Now, early-adopter broadcasters have recognized the value of object storage. They are asking vendors to integrate with cloud storage, which, in effect, is a request to integrate with object-storage technology. Whether the cloud-storage business model is right for broadcasters or not, software vendors can use the same Amazon S3 interface for cloud storage as they do for object storage (the same technology as cloud storage, but deployed where you want it).

While early adopters are excited about cloud storage, they have concerns about security, governance, bandwidth, and access costs. Whether in the cloud or on-site, media operations are drawn to a storage platform that lets them control the three most difficult data-related problems they face today:

  • Data management
  • Data migration
  • Data mobility

Therein lies the strength of object storage.

In an infrastructure based on object storage, you locate a piece of content simply by pulling it from a single object store — but having one object store doesn’t mean you have one particular storage array in one place. We all know that storage is made up of different platforms, possibly across several different geographies, with different levels of storage performance for different workflows in different parts of your operation. Object storage doesn’t change that; it simply allows it to look like a single repository to the outside world.

If you need multiple copies of content, the object store knows where they are. If a particular copy is no longer needed, object storage can remove it. With object stores, you set policies for your content so that copies and deletions are automatic. This automation is critical because when you have millions of files, if deletions don’t happen automatically, then they typically don’t happen at all, and your storage gets fuller and fuller.

Object storage lets each asset management system in your operation support the various workflows it does best rather than managing the file systems of many different storage platforms. Unique asset identifiers make it simple for your asset management systems to present the right content to the users who need it.

To ensure that your data is always available, some object stores have fault-tolerant architectures for uninterrupted operations, including during upgrades and infrastructure refreshes.

To keep your data intact over time, some object stores use digital fingerprinting, tamper detection, integrity verification, and automatic regeneration of suspect objects — not just on ingest, retrieval, replication, and migration, but also with objects that have gone unused for years.

Your content can gain or lose value over time. High-value content needs high-performance storage to keep it readily available, while low-value content can sit on lower-performance storage such as tape or cloud. An object store that can archive to tape and cloud using its native policy engine will automatically move content to the appropriate storage tier, thereby ensuring the right performance-to-cost ratio for that content’s value:

  • Storage for active archives
  • Tape or cloud storage for near-line access to less valued content
  • Tape storage for long-term preservation

Collaboration, localized delivery, and cultural conformity often require you to store the same asset in more than one place. Some object stores can copy or move content automatically — either upon ingest or sometime later — based on the policies you set and triggered by metadata. As a result, you no longer have to spend hours copying every asset that is tagged in a certain way. Instead you can simply create a new policy associated with that tag, and the object store will automatically copy those objects to a second location.

For example, if yours is a news organization with footage of a natural disaster, you probably want every one of your news bureaus worldwide to have a copy of that content. The right object store can make it happen automatically, tracking where each copy goes and managing it when it gets there. Then, when that content is no longer relevant to the international bureaus, the same object store can delete all of the copies. 

In the diagram above, content is stored based on user-driven policies triggered by metadata. Users can create a new policy associated with a tag, and the object store will automatically copy those objects to a second location.

In the diagram above, content is stored based on user-driven policies triggered by metadata. Users can create a new policy associated with a tag, and the object store will automatically copy those objects to a second location.

With all of these advantages, object storage can organize your storage infrastructure in a way no other storage technology can. Using policies to manage objects across data tiers, locations, resiliency schemes, and retention periods, object stores manage your media across space and time.

Jason Danielson, Media and Entertainment Solutions, Product and Solution Marketing, NetApp

Jason Danielson, Media and Entertainment Solutions, Product and Solution Marketing, NetApp

You might also like...

HDR & WCG For Broadcast: Part 3 - Achieving Simultaneous HDR-SDR Workflows

Welcome to Part 3 of ‘HDR & WCG For Broadcast’ - a major 10 article exploration of the science and practical applications of all aspects of High Dynamic Range and Wide Color Gamut for broadcast production. Part 3 discusses the creative challenges of HDR…

IP Security For Broadcasters: Part 4 - MACsec Explained

IPsec and VPN provide much improved security over untrusted networks such as the internet. However, security may need to improve within a local area network, and to achieve this we have MACsec in our arsenal of security solutions.

Standards: Part 23 - Media Types Vs MIME Types

Media Types describe the container and content format when delivering media over a network. Historically they were described as MIME Types.

Building Software Defined Infrastructure: Part 1 - System Topologies

Welcome to Part 1 of Building Software Defined Infrastructure - a new multi-part content collection from Tony Orme. This series is for broadcast engineering & IT teams seeking to deepen their technical understanding of the microservices based IT technologies that are…

IP Security For Broadcasters: Part 3 - IPsec Explained

One of the great advantages of the internet is that it relies on open standards that promote routing of IP packets between multiple networks. But this provides many challenges when considering security. The good news is that we have solutions…