Archiving In The Digital Age

In 2007, it was estimated that the planet had stored 275 zettabytes (or 10 to the 1021) of data - the equivalent of a stack of CDs reaching from earth to beyond the moon! And in a single generation, our planet will add more information than in all of recorded history, some 40 trillion gigabytes of digital information. But, that data is delicate and easily lost.

Across the globe, the preservation and importance of archives has risen in prominence due to two important factors. The first is when archives and the archivist are under most scrutiny during moments in history when retrospection is at its most important. National anniversaries, independence celebrations, major political upheavals; across the world archivists are watching a ticking clock toward the moment when the eyes of the nation will be on them to deliver that speech from half a decade earlier or that image of a young child who later would lead a nation to independence.

The other important factor about archives is the continual degradation of the archive media. In an irony of our age, as technology has allowed more information to be stored in greater density, in some aspects, the durability of the medium has declined. Where an inscribed stone tablet from 2000bc Babylon will still be legible today, a 2 1⁄2-inch print from the 1890 will almost certainly be in a state of decay that will render it unrecognizable within a century. 

Many of the 20 or so tape formats developed for audio and video since the 1950’s are now reaching the point where the magnetic encoding is failing as is the physical condition of the cartridges. Even the modern hard disk using electrometric plates, with a storage capacity equivalent to 10 million pages while spinning, will only last a decade at best.

Historically, records were something physical that a human could see and hold. Finding records was often a long and manual process. Image courtesy keptarts.brighton.ac.uk

Historically, records were something physical that a human could see and hold. Finding records was often a long and manual process. Image courtesy keptarts.brighton.ac.uk

The cause of damage

Even climate controlled and highly sterile environments can’t sustain forever the magnetic patterns and chemical compound in film stock that will eventually eat away at the material. The danger is not just ageing. In one of the largest studies of recent times; in 1995 the United Nations Educational, Scientific and Cultural Organization (UNESCO) and International Federation of Library Associations and Institutions (IFLA) gathered information on 6,250 archives and repositories from around the world and found that over 95% had reported significant losses during the previous 50 years. In closer detail; armed conflict, fire, flood, civil disorder and poor storage conditions topped the list of causes of damage.

Yet, it is video that is set to dominate the next century. YouTube, as the world’s most popular video-on-demand website, uploads roughly 4000 hours of video each hour and other popular sites such as DailyMotion and the Chinese site Youku are quickly catching up in terms of viewable material. However, the most valuable archives are still national repositories, often built on less commercial grounds as guardians of history for education and cultural enrichment.

Media degradation

Often working in close co-operation with state broadcasters, over the last 10 years many archives such as British National Archives have added more video content than all of the combined texts collected in the previous century. For every document or video viewed in its reading room in London, a further 200 are distributed via the internet. This is a result of a major digitisation project which started in 2008 that has helped it to offer a better service, to more researchers, historians and filmmakers while crucially reducing costs. Yet the British Archive, like many others in Singapore and Malaysia, has undertaken digitisation and computerisation projects to meet the bigger threat of media degradation.

Before restoration can even begin, the restorer needs to do the groundwork which starts with an investigation of the condition of archive storage such as temperature, humidity and light. The stored metadata attached to each item needs to be understood and catered for in the restoration processes and each tape item needs careful inspection.

Cleaning and preparation

These steps are vital as some restoration processes, although potentially providing the best results, have risks which must be assessed and weighed up by the owner of the content. In some cases, more cumbersome and labour intensive processes need to take place to minimise potential damage. Sometimes tapes may need to be baked in a special oven to remove contaminants with each batch requiring different temperatures, duration and cooling conditions to prepare for digitisation.

The careful cleaning and preparation for playback often needs to take place using custom modified equipment that reduces the strain on fragile stock. In fact, a single reel or tape may go through a 30 man-hour process at the hands of several specialists before it even gets to a digitisation step.

Even well-funded archives rarely do these processes in house as the skills and technical equipment are both costly to obtain and challenging to maintain. Restoration of media is not a subject taught on most university syllabuses - yet the need is there!

As the ticking clock of media degradation intersects with upcoming national anniversaries and increasing requests for access to the archive, archivists are seeking help. Ultimately, there are more unrestored and deteriorating assets in national archives than there are people with the skills to preserve the content – and time is running out!

Spectra Logic T-Finity represents current long-term storage technology

Today's large and long-term storage platforms rely on digital technology. Without the proper technical support, all those records cannot be recovered. While reliable, how long will the media continue to use physical tape and mechanics to store humanity's history?

You might also like...

Designing IP Broadcast Systems - The Book

Designing IP Broadcast Systems is another massive body of research driven work - with over 27,000 words in 18 articles, in a free 84 page eBook. It provides extensive insight into the technology and engineering methodology required to create practical IP based broadcast…

Demands On Production With HDR & WCG

The adoption of HDR requires adjustments in workflow that place different requirements on both people and technology, especially when multiple formats are required simultaneously.

If It Ain’t Broke Still Fix It: Part 2 - Security

The old broadcasting adage: ‘if it ain’t broke don’t fix it’ is no longer relevant and potentially highly dangerous, especially when we consider the security implications of not updating software and operating systems.

Standards: Part 21 - The MPEG, AES & Other Containers

Here we discuss how raw essence data needs to be serialized so it can be stored in media container files. We also describe the various media container file formats and their evolution.

NDI For Broadcast: Part 3 – Bridging The Gap

This third and for now, final part of our mini-series exploring NDI and its place in broadcast infrastructure moves on to a trio of tools released with NDI 5.0 which are all aimed at facilitating remote and collaborative workflows; NDI Audio,…