Archiving In The Digital Age
Video engineers may recall tape libraries were like this film storage. Finding something was difficult. Image courtesy Irish Film Institute
In 2007, it was estimated that the planet had stored 275 zettabytes (or 10 to the 1021) of data - the equivalent of a stack of CDs reaching from earth to beyond the moon! And in a single generation, our planet will add more information than in all of recorded history, some 40 trillion gigabytes of digital information. But, that data is delicate and easily lost.
Across the globe, the preservation and importance of archives has risen in prominence due to two important factors. The first is when archives and the archivist are under most scrutiny during moments in history when retrospection is at its most important. National anniversaries, independence celebrations, major political upheavals; across the world archivists are watching a ticking clock toward the moment when the eyes of the nation will be on them to deliver that speech from half a decade earlier or that image of a young child who later would lead a nation to independence.
The other important factor about archives is the continual degradation of the archive media. In an irony of our age, as technology has allowed more information to be stored in greater density, in some aspects, the durability of the medium has declined. Where an inscribed stone tablet from 2000bc Babylon will still be legible today, a 2 1⁄2-inch print from the 1890 will almost certainly be in a state of decay that will render it unrecognizable within a century.
Many of the 20 or so tape formats developed for audio and video since the 1950’s are now reaching the point where the magnetic encoding is failing as is the physical condition of the cartridges. Even the modern hard disk using electrometric plates, with a storage capacity equivalent to 10 million pages while spinning, will only last a decade at best.
Historically, records were something physical that a human could see and hold. Finding records was often a long and manual process. Image courtesy keptarts.brighton.ac.uk
The cause of damage
Even climate controlled and highly sterile environments can’t sustain forever the magnetic patterns and chemical compound in film stock that will eventually eat away at the material. The danger is not just ageing. In one of the largest studies of recent times; in 1995 the United Nations Educational, Scientific and Cultural Organization (UNESCO) and International Federation of Library Associations and Institutions (IFLA) gathered information on 6,250 archives and repositories from around the world and found that over 95% had reported significant losses during the previous 50 years. In closer detail; armed conflict, fire, flood, civil disorder and poor storage conditions topped the list of causes of damage.
Yet, it is video that is set to dominate the next century. YouTube, as the world’s most popular video-on-demand website, uploads roughly 4000 hours of video each hour and other popular sites such as DailyMotion and the Chinese site Youku are quickly catching up in terms of viewable material. However, the most valuable archives are still national repositories, often built on less commercial grounds as guardians of history for education and cultural enrichment.
Media degradation
Often working in close co-operation with state broadcasters, over the last 10 years many archives such as British National Archives have added more video content than all of the combined texts collected in the previous century. For every document or video viewed in its reading room in London, a further 200 are distributed via the internet. This is a result of a major digitisation project which started in 2008 that has helped it to offer a better service, to more researchers, historians and filmmakers while crucially reducing costs. Yet the British Archive, like many others in Singapore and Malaysia, has undertaken digitisation and computerisation projects to meet the bigger threat of media degradation.
Before restoration can even begin, the restorer needs to do the groundwork which starts with an investigation of the condition of archive storage such as temperature, humidity and light. The stored metadata attached to each item needs to be understood and catered for in the restoration processes and each tape item needs careful inspection.
Cleaning and preparation
These steps are vital as some restoration processes, although potentially providing the best results, have risks which must be assessed and weighed up by the owner of the content. In some cases, more cumbersome and labour intensive processes need to take place to minimise potential damage. Sometimes tapes may need to be baked in a special oven to remove contaminants with each batch requiring different temperatures, duration and cooling conditions to prepare for digitisation.
The careful cleaning and preparation for playback often needs to take place using custom modified equipment that reduces the strain on fragile stock. In fact, a single reel or tape may go through a 30 man-hour process at the hands of several specialists before it even gets to a digitisation step.
Even well-funded archives rarely do these processes in house as the skills and technical equipment are both costly to obtain and challenging to maintain. Restoration of media is not a subject taught on most university syllabuses - yet the need is there!
As the ticking clock of media degradation intersects with upcoming national anniversaries and increasing requests for access to the archive, archivists are seeking help. Ultimately, there are more unrestored and deteriorating assets in national archives than there are people with the skills to preserve the content – and time is running out!
Spectra Logic T-Finity represents current long-term storage technology
Today's large and long-term storage platforms rely on digital technology. Without the proper technical support, all those records cannot be recovered. While reliable, how long will the media continue to use physical tape and mechanics to store humanity's history?
You might also like...
Expanding Display Capabilities And The Quest For HDR & WCG
Broadcast image production is intrinsically linked to consumer displays and their capacity to reproduce High Dynamic Range and a Wide Color Gamut.
Standards: Part 20 - ST 2110-4x Metadata Standards
Our series continues with Metadata. It is the glue that connects all your media assets to each other and steers your workflow. You cannot find content in the library or manage your creative processes without it. Metadata can also control…
Delivering Intelligent Multicast Networks - Part 2
The second half of our exploration of how bandwidth aware infrastructure can improve data throughput, reduce latency and reduce the risk of congestion in IP networks.
If It Ain’t Broke Still Fix It: Part 1 - Reliability
IP is an enabling technology which provides access to the massive compute and GPU resource available both on- and off-prem. However, the old broadcasting adage: if it ain’t broke don’t fix it, is no longer relevant, and potentially hig…
NDI For Broadcast: Part 2 – The NDI Tool Kit
This second part of our mini-series exploring NDI and its place in broadcast infrastructure moves on to exploring the NDI Tools and what they now offer broadcasters.