The Digital Challenge - New perspectives for the Metamorfoze Programme
Dennis Schouten, Programme manager of Metamorfoze, Research & Development Division, Koninklijke Bibliotheek, P.O. Box 90407, 2509 LK The Hague, Netherlands, firstname.lastname@example.org
2007 is an important year for the programme. The budget has increased substantially, which allows the programme to be carried out on a much larger scale. Government archives are now included in the programme as well, and, last but not least, Metamorfoze is shifting towards digitization as a conversion method for mass-preservation projects instead of microfilming.
This paper will give an overview of these new developments and plans for the future, but it will start with a short retrospective.
Metamorfoze is the national programme for the preservation of the Dutch paper heritage. The programme mostly concerns itself with mass preservation of paper. It deals with various forms of autonomous decay that can effect paper such as paper acidification, and ink and copper corrosion. In the 1980’s and 1990’s research was carried out by the (KB, National Library of the Netherlands) and the (National Archives) to assess the scale of this problem. An inventory was made of all the paper heritage material that was considered of national importance. Eventually, a programme plan was drafted based on this inventory and the research that had been done. The Metamorfoze programme was launched in 1997. It is now in its third phase: 2005-2008.
Metamorfoze is financed by the Dutch (Ministry of Education, Culture and Science). The KB and the Nationaal Archief cooperate in the Metamorfoze programme, which is coordinated by the National Preservation Office (NPO), housed at the KB. The NPO’s main task is to coordinate preservation projects in Dutch heritage institutions by supporting the institutions in preparing project proposals, providing consultation when needed, and by carrying out the projects themselves. It also includes quality management of products, providing publicity about the programme and further preservation research. The programme subsidises preservation programmes in Dutch heritage institutions. The granting of subsidies is the responsibility of an advisory board, which consists of 7 independent representatives from different heritage and scientific institutions, who together share a broad view on Dutch culture and history.
First of all, conversion is applied as a preservation method. This means that the content of the threatened material is transferred to another storage medium by means of either microfilming or digitization. When the programme started in 1997, microfilming was a reliable method to preserve the content of an endangered document. From 2001 on digitization was also applied, sometimes in combination with microfilming. Preserved collections were presented on the website of (Memory of The Netherlands).
Besides conversion, Metamorfoze is also concerned with preserving the original documents themselves. Treatment of the originals through small repairs, deacidification, and ink and copper corrosion treatments are subsidised. All originals are stored in acid-free wraps and boxes in a controlled environment, and protected from use.
In its third phase, from 2005 on, Metamorfoze is being financed on a structural basis by the Ministry of Education, Culture and Science. Furthermore, the KB started to cooperate with the Nationaal Archief in coordinating the programme. Staff members of the Nationaal Archief were seconded at the National Preservation Office to coordinate the preservation of large governmental archives. After an introductory pilot phase in 2006, archives are now being preserved on a large scale by Metamorfoze as well. In the first two phases of Metamorfoze, the programme focused on the problem of acidification of paper which mainly occurs in paper produced between 1840 and 1950. The new archives programme introduced entirely new forms of autonomous decay into Metamorfoze, such as ink and copper corrosion, which mostly occur in paper from before the 1840-1950 period. Ink and copper corrosion constitute one of the main conservation problems archival institutions have to deal with. For Metamorfoze it meant that paper from before 1840-1950 was eligible for subsidies for conservation and conversion as well.
Through extended funding in 2007 the realization of all sections of the programme (collections, archives, books, periodicals and newspapers) could be sped up and the scope of the different sections could be extended. With the present budget of six million euros a year for preservation subsidies, Metamorfoze faces a great challenge.
The scaling up of the programme does not pose the only challenge for Metamorfoze. The introduction of digitization as a conversion method for mass preservation purposes may have an even greater impact on the organisation of the programme. Metamorfoze has gained experience with digitization projects since 2001. From that time on the possibility was created to preserve and digitize literary collections and to present the digitized material on the website of the Memory of the Netherlands. In 2004-5 Metamorfoze carried out a pilot project for the preservation and digitization of 4 Dutch newspapers. They were subsequently made available on the web with a full-text search engine.
From 2004 on most heritage institutions participating in the programme preferred digital user copies over microforms, and Metamorfoze decided to subsidize these digital copies as well. However, in all of these cases the microfilm served as the preservation copy. In most cases access scans were made by digitizing the duplicate microfilms. In a few cases, however, mostly involving full colour originals, these were both scanned and put on microfilm.
Nowadays Metamorfoze is facing a new paradigm shift: microfilming is being entirely abandoned as a preservation method and digitization is being introduced as a conversion method for both preservation and access. Undoubtedly, digitization has many advantages over microfilming on the access side: digital images include colour reproduction, they allow remote access and they can be easily searched by using OCR (Optical Character Recognition). From the point of view of preservation, digitization offers exciting possibilities as well. Since digitized copies contain colour, a high quality digital image is closer to the original than a microfilm will ever be.
There are nevertheless a number of issues to be tackled before digitization can definitively be used as a conversion method for preservation and access. Standards and guidelines, the workflow, the metadata, long-term storage and retrieval of digital images all have to be developed and dealt with. Metamorfoze is now in the midst of these developments.
In June 2007 Metamorfoze published the draft version of the Metamorfoze Preservation Imaging Guidelines (). These guidelines are based on two main principles: firstly, all information in the original must be visible in the digital surrogate and secondly, the quality of the image must be objectively measured with the help of specific software instead of merely using visual inspection. The guidelines refer to a number of image characteristics: tonal reproduction, noise, signal-to-noise ratio, illumination, colourcast, colour reproduction, detail reproduction, geometrical reproduction and image artefacts. In order to allow for an optimal and objective inspection of the quality of the image, several technical targets are being added to each capture: Kodak greyscale, Macbeth colour checker, QA 62-Slanted edge MTF target and an illumination sheet. For quality inspections the following software is being used: Adobe photoshop, ImCheck (developed by Peter Burns, Kodak) and IE Analizer (Image Engineering, Cologne, Germany).
In August of 2007 an instruction session was organized at the KB for vendors on producing images according to the preservation imaging standards. At this moment the quality managers of Metamorfoze are working at fine-tuning and benchmarking the cameras and scanners of those vendors which are interested in participating in preservation imaging projects. However, some further testing will be necessary. The English translation of the guidelines has been sent to a number of image scientists in several countries for peer reviews. By using software in the quality management process, it can be automated and thus create a more efficient workflow. Again further research and development will be necessary and is scheduled for the beginning of 2008.
Long-term storage and costs
Metamorfoze is responsible for the storage of the master images, both analogue and digital. The KB has created an infrastructure, the , to guarantee long-term storage of digital masters. An important aspect of preservation imaging is the cost of long-term storage and retrieval of the master images. Cold storage of traditional microfilms is relatively simple and cheap, compared to the cost of storing digital images. Recent calculations of the costs of a storage facility for the millions of high-quality TIFF images Metamorfoze will produce each year, indicate that storage would take up an unacceptably large part of the Metamorfoze budget.
Right now, research is being carried out to establish whether it would be justified to abandon the TIFF-standard for master images and shift to another file format, for instance lossless compressed JPEG2000 or PNG. When using these file formats there would be hardly any loss in quality and the storage costs would decrease substantially. Simultaneously, other research is being conducted to establish what the exploitation costs of long-term storage of digital images per terrabyte per year exactly are. The results of both research projects will be available by the end of 2007.
Since 2006 Metamorfoze has subsidised a number of smaller pilot projects to gain experience with the new techniques and workflow that are involved with preservation imaging. Most of these projects were concerned with collections or archives which contain many full-colour images such as children’s picture books, maps and graphs. The experiences that were gained during these pilot projects were used when drafting the preservation imaging guidelines. In October 2007 a more substantial pilot project will be launched: digitizing an important Dutch art magazine called De Gulden Winckel.
Subsequently, at the end of 2007 the Metamorfoze advisory board will probably grant the first subsidies for real preservation imaging projects, to be carried out in 2008. Metamorfoze does not prescribe preservation imaging. If heritage institutions prefer using an analogue intermediate (microfilm) to be digitized, this will be subsidised as well. It is nevertheless to be expected that heritage institutions in The Netherlands will rapidly embrace the new method because of its many advantages.
To be continued…
At this moment the Metamorfoze programme is half way. It has preserved two-thirds of the Dutch cultural heritage collections of important national and international value. As of 2007 Metamorfoze shifting its focus to the preservation of books and periodicals, because of the vast amounts of material which still has to be preserved. Research is being carried out to establish the total volume of archives of national importance which are suffering from autonomous decay.
Metamorfoze is becoming a mass-digitization programme, and as such it will be coordinated with other large mass-digitization programmes in the Netherlands and internationally. An example is the separate mass-digitization programme for newspapers which was set up by the KB. In light of this programme, Metamorfoze has halted its own activities with regard to newspapers and cooperates with the KB.
There will be an inevitable shift towards access (OCR, text retrieval) through preservation imaging, but, of course, the programme will continue to focus on autonomous decay and the preservation of our paper heritage. Calculations have indicated that according to the selection and strategy plan adopted in 1996, the Metamorfoze programme should be able to conclude its activities in 2016.
Web sites referred to in the text