Web-Mapping Applications for Accessing Library Collections: Case Studies using ESRI’s ArcIMS at the National Library of Scotland
For several decades, map libraries and our users have appreciated the value of graphical search methods, and the integrating power of geographical coordinates for spatial searching (Van de Waal, 1974; Larsgaard, 1978). More recently there have been developments in software and data availability to allow the more widespread practical implementation of web-based graphical search tools, using dynamic maps and geo-referenced records. This year, the (NLS) has implemented pilot web-based applications using ESRI’s ArcIMS software, to provide an online spatial search method for NLS collections. This paper describes the background to these applications, the software and hardware used, interfaces developed, and I will end with an examination of the problems and prospects of this web-mapping technology.
A crucial impetus to the project was the Ordnance Survey’s Pilot Pan-Government Agreement, from April 2002, which became a 3-year (PGA) from April 2003. For the first time, a portfolio of OS data products (including maps, boundary files, and gazetteers) were made available for free use in UK government institutions, provided that these institutions used the OS products to support their core business objectives. The broad definition of government institution, which included Non Departmental Public Bodies such as NLS, allowed us to participate in the scheme, using the OS products to support our key objective of promoting access to our library collections. Although we could have developed these applications without OS mapping, this would have been difficult and costly, and the PGA provided a crucial lever within NLS to gain support for pilot projects to explore the potential of web-mapping technology.
The project was also assisted by technological developments. In the last few years, many commercial GIS vendors have been actively developing modules that allow a growing range of server-side web-mapping functionality, and the prices for these have been steadily falling. The widespread uptake of web-mapping applications, or web-enabled GIS, have also lead to a large pool of programming expertise in developing these tools, and a greater choice of methods for implementing these tools using different applications and programming languages. The growing uptake of the Internet, and the increasing share of higher-bandwidth connections have also provided key prerequisites to these applications. This in turn has fostered a growing web-mapping literacy amongst Internet users, a trend that will surely increase, creating a growing expectation of the need to provide search methods using dynamic maps.
Within NLS our new strategic and corporate plans in 2003-4 placed a strong emphasis on promoting access, especially through expanding the virtual NLS, and developing innovative ‘socially inclusive’ applications. Web-mapping applications fitted well into these corporate aims, and the expansion and promotion of GIS in related government institutions, ‘joining’ public sector activities, promoting e-government services, as well as in digital libraries provided other useful exemplars and case studies. The popular and expanding digital library of historical map images at NLS had also provided a useful base of core skills in this area, and a willingness to build on these applications.
Software and hardware
Following advice from the OS PGA consultants, a software review, and a recognition that through the some software could be purchased at a significant discount (see: ), we decided to use ESRI’s ArcGIS and ArcIMS software. The main reasons for this were:
|·||Along with some other GIS vendors, ESRI’s global reach and market dominance, suggested an ongoing likelihood of future survival and product enhancement.|
|·||products were by far the leading choice of software within Scottish government institutions, and therefore there was a wide body of experience and applications we could draw on for advice and support.|
|·||Our informal links with the Royal Commission on the Ancient and Historical Monuments of Scotland ( ), whose CANMAP application used ArcIMS provided an early and local exemplar of what we intended to do.|
|·||ESRI had a local office in Leith which could provide training and support, backed up by extensive online documentation, support, virtual campus training etc.|
|·||ESRI software could include LizardTech MrSID images, as used on the existing NLS Digital Library|
ESRI’s ArcIMS 4.0 was purchased in September 2003, and installed on a web-server workstation with the following basic specifications:
- Processor: Xeon 2.4 GHz
- RAM - 1Gb
- Storage: 2 x 80Gb hard disks
- Operating System: Windows 2000
The main related software for the web-application (Apache Tomcat, Java server, and Coldfusion MX) was already in use on the NLS website. ArcGIS 8.0 also was purchased in September 2003 and installed in the Map Collections division for back-end work preparing geographic data, including maps and gazetteers, primarily from OS.
As the ArcIMS and ArcGIS software was purchased through CHEST, its combined cost was £2,900 (incl. VAT), less than one third of its standard commercial price, whilst the web-server cost was £2,700 (incl. VAT).
Our specific requirements were to gain enough knowledge of the software to develop and maintain a fairly specific application, relatively limited in terms of GIS potential. Rather than attending a range of general ArcGIS and ArcIMS training courses, and then build the application from scratch, purchasing consultancy support where required, we tendered for a “Training and Consultancy” package, integrating these essential requirements at a much lower cost. Expressions of interest were received from three companies, over the summer of 2003, all of whom had extensive experience of ESRI software development and training. Their proposals varied between about 10-20 days of paid consultancy work, including both offsite software development and onsite training and implementation, and in the end, we chose Forth Valley GIS, based in Stirling to develop our application. Work began in October 2003, and although other priorities both at NLS and Forth Valley delayed the original timetable, the main application was delivered by early March 2004.
The main theoretical model underlying the work, was the concept of using geographical coordinates as the integrative common element between textual and graphical search methods, and geographic resources held by NLS, that had some implicit geographical locational reference. In our case, gazetteers of place-names and postcodes, as well as modern mapping at a range of scales on the left hand side of the diagram below, were provided by OS. We also obtained boundary files of historical counties and parishes through collaborative work with the allowing these to be used for searching and geo-referencing.
Figure 1. Illustration of geo-referenced search and retrieval in NLS using OS databases
Whilst recognising the broader, long-term potential of this technology in extending across thousands or even millions of records in NLS and beyond, successful, short-term applications would need to focus on limited, subject-specific collections. Our initial pilot application therefore focused on NLS digital images covering the geographic area of Aberdeenshire (about 600 altogether). The existence of these records in an Microsoft Access database, with geographic, text-based indexing that could be readily fitted to counties, parishes, burghs, or specific point locations, made the geo-referencing relatively straightforward. Following development of the pilot application and in-house geo-referencing work extending it to cover all of Scotland, two further applications have been developed:
1. Bathymetrical Survey of the Freshwater Lochs of Scotland, allowing the location of 562 lochs to be spatially presented and queried, tied into a new website devoted to the Bathymetrical Survey
2. Ordnance Survey National Grid maps of Scotland, allowing the records for large-scale Scottish post-war mapping to be geographically selected.
Interface design and functionality
Although intended for different users and collections, all three applications share the same basic interface and mapping, shown in Figure 2.
Figure 2: Opening screen interface
As always, the interface design reflects the need to address some conflicting pressures:
|·||Simplicity and complexity: the need to keep instructions and presentation clear and brief, whilst allowing a range of search options and approaches.|
|·||Small and large screens: the need to present a lot of information in a small space encouraged the use of left-hand panels that could be toggled, and a basic decision to design for 800 x 600 resolution screens or higher.|
|·||Older and newer browsers: designing the site for both Internet Explorer, Netscape, and Mozilla for current and older versions requires extra programming, and so the application is only fully functional in IE 5.5 or later, and Netscape 6.0 or later.|
The following OS mapping was loaded to the server:
|·||1:250,000 scale colour raster|
|·||1:50,000 scale colour raster|
|·||1:25,000 scale colour raster|
|·||1:10,000 scale colour raster|
As TIFF images, these took up about 19 Gb of space collectively, and with their associated TIFF for World files (*.tfw), they were presented as seamless layers of mapping by ArcIMS, in our case as portable network graphic (*.png) images.
The OS 1:50,000 Landranger gazetteer and CodePoint postcode files, along with the county and parish listings were all loaded into separate MS Access databases. The querying of these is all through Coldfusion, with the map window zooming to display the location of selected entries.
The selection of records on the map, by clicking and dragging a window for an area/mulitple selection, or using an Identify tool for point/item selection, presents brief lists of record titles in the left-hand panel, shown in figure 3.
Figure 3 The selection of brief titles of records
The selection of these presents the record (with an image thumbnail), and in the case of map records, these images can be zoomed using the MrSID Express Server software, used for the Digital Library maps on the NLS website.
|·||Cross-browser compatibility. The application was originally written for Internet Explorer, version 6.0 or higher, and further assistance from Forth Valley was required during July 2004 to develop the navigation tools so that everything worked satisfactorily across other browsers.|
|·||Downtime. We experienced problems with the connector between Coldfusion and ArcIMS, whereby the application failed after about 10-20 hours. A batch program was set up to stop and start the ArcIMS web services twice-daily, providing full 24/7 access, and it is hoped a future release of the connector may resolve this problem.|
Unfortunately, the size of regularly updated raster map images during panning and zooming, create limitations to the usability of this application by low-bandwidth Internet users.
We hope to undertake more active user testing and feedback on the interfaces, although some of the problems of screen design and functionality are not easily resolved. The main ongoing plans are expanding the scope of existing applications, and creating new, related applications for other geographically distributed collections. This will allow some of the problems of scalability and record categorisation in georeferenced digital libraries () to be addressed on a bottom-up, ongoing basis.
Through the cropping and georeferencing of scanned images of historical maps, the presentation of zoomable layers of seamless historical mapping becomes possible, both as map information in itself, and as a searchable spatial backdrop to other resources and information. The development of these will hopefully make us less dependent of OS mapping in future.
One of the main challenges to the expansion of this technology to wider library collections, is the appropriate georeferencing of these collections. We also hope to explore more automated methods for georeferencing, such as using Geoparsing technology ().
Larsgaard, M.L.: Map librarianship: an introduction. Libraries Unlimited, 1978.
Waal, H. Van de: “The application of geographical co-ordinates for the retrieval of maps in a computerized map-catalogue”. International Yearbook of Cartography, 14(1974), 166-172.
Web sites referred to in the text
See, for example, the pioneering influence of the Alexandria Digital Library < http://www.alexandria.ucsb.edu/>. The 'Georeferencing in Digital Libraries', Special issue of D-Lib Magazine, 10(2004)5 < http://www.dlib.org/dlib/may04/05contents.html> discusses several other recent applications, including the Electronic Cultural Atlas Initiative, and the EDINA digital mapping projects.
These web-mapping applications have been developed collaboratively with the NLS Web Development Officer, Tony Stuart, to whom I am very grateful.
In the Summer of 2003, we estimated that the total cost of two people attending the Introduction to ArcGIS, Introduction to ArcIMS and Customising ArcIMS training courses was about £4,000; 10-days of ESRI consultancy at £760 per day would have taken combined costs to over £12,000. Our training and consultancy package cost less than half this total.