Special collections and archives held by research libraries have long been recognised as cultural assets to their institutions with strong research and educational potential. Yet, over the past decades, aspects of a fast changing society, such as the digital revolution, and the challenging economic climate have greatly shaped the practices and values of academic and cultural heritage institutions. Many institutions, including research libraries, have been called to respond to the call for openness in scholarship and culture as well as to prove their worth and positive impact on society.
According to Streatfield and Markless (2012, pp. 7–12), factors such as change caused by recent developments in Information and Communications Technology (ICT) and challenges imposed by the current economic climate have resulted in a cultural shift that involves a greater ‘focus on performance management and accountability in public institutions’, the development of a ‘value for money ethos’ that is linked to the distribution of funding based on results, and a shift towards more evidence-based work as a response to the need of navigating and managing change. These factors along with the introduction of assessment systems, such as the Research Excellence Framework (REF)1 in the UK Higher Education agenda, have had a significant effect on research libraries which has led to the employment of more user-centred approaches for developing their services as well as to the introduction of practices and methods for collecting and demonstrating their impact.
This paper is based on a Research Libraries UK (RLUK) project looking at the approaches libraries from across the RLUK membership employ for achieving and capturing the impact of collection-based services and activities in both the physical and digital environment. Currently, in the Covid-19 era, the priorities of cultural heritage institutions, including research libraries, shift and greater effort is placed on delivering activities and engaging with communities online; capturing and measuring the impact of these initiatives will be an important part of the strategic goals of institutions in the near future. For the purposes of this paper, the focus is placed on the results of the project related to the opportunities and challenges that the digital age presents for increasing and evidencing the value of research library collections. Although the project was conducted before the Covid-19 pandemic, many of the key points and recommendations will hopefully be of interest to institutions adapting their practices and strategies to continue supporting their communities and demonstrating the value of their collections in today’s challenging times.
Previous studies have attempted to map the cultural shift that special collection departments have been experiencing as part of the digital revolution as well as the need to develop new business models. Cullingford (2016) provided a holistic view of the matters that concern special collection professionals nowadays, from care and digitisation of collections to widening access and fundraising. Tam (2017) explained how digitisation and outreach activities have opened up collections and, consequently, transformed the image of the profession and its practices; from ‘treasure rooms’ with limited access, special collections now increasingly form the basis for the development of services and programmes that enable research libraries to engage with their diverse audience groups in new and creative ways.
The reports by RLUK (Cullingford, Peach, & Mertens, 2014) and OCLC (Dooley et al., 2013), in collaboration with RLUK, apart from exploring issues around the development of special collections by modern research libraries in the UK and Ireland, have highlighted the important role that unique and distinctive collections play in their institutions and the ways these can foster innovation and creativity. The activities led by special collections and archives professionals to engage different audience groups, such as faculty, students or the general public have been further investigated by various studies (e.g. see Harris & Weller, 2012; Mason, 2014; Schmiesing & Hollis, 2002).
Others have focused particularly on the effects of digitisation on special collections as well as the advantages and challenges concerning their creation and management in the digital age. Tanner (2012a) outlined the opportunities that have arisen through digitisation and the positive impact that activities such as making available digitally and virtually reunifying collections can have on research, teaching and learning, and community engagement. He also highlighted the economic benefits that digitised resources can bring to the UK and beyond by ‘delivering efficiency, innovation and enhanced skills, by underpinning competitiveness, and by developing a UK brand for universities and higher education worldwide’ (Tanner, 2012a, p. 111). According to Daigle (2012), one of the biggest advantages of making special collections available digitally is the opportunity to present material in dynamic and innovative ways that enhance its uniqueness while also enabling effective care and management.
However, the new information landscape has brought not only opportunities, but also challenges for institutions and the profession. Regarding digitisation of collections, Prescott (2012) argued that it can constitute a challenge for institutions and a hindrance in the path towards achieving impact in the digital age when its organisation is based on projects and not on workflows; the importance of focusing on workflows lies in the fact that, in that way, processes can be transferable and the road to impact more fruitful. Creating and managing digital collections for the long term can also be an expensive and time consuming process that requires funding and other resources difficult to obtain in times of economic downturn; this can put additional pressure to institutions and their staff to evidence the use and showcase the impact of collections (also in Hughes, 2012).
In the face of constant budget reductions and pressure to demonstrate the impact of library services and initiatives to funders and other stakeholders, advocating for the value of special collections and archives and making their potential visible is an essential step for ensuring that their significance is acknowledged and awarded. Thus, it is not surprising that recent years have seen the publication of several impact models related to cultural heritage resources; Tanner’s ‘Balanced Value Impact Model’ (Tanner, 2012b) and Europeana’s ‘Impact Playbook’ (Verwayen et al., 2017) are two well-known examples.
Moreover, much discussion has focused on the most appropriate ways of evidencing the impact of resources, such as special collections in both physical and digital format (e.g. see Dupont & Yakel, 2013; O’Gara, Walters, & Putirskis, 2010). According to Carter (2012) and Dupont and Yakel (2013), improving access to special collections and employing more audience-focused strategies are pathways to impact and can lead to activities that produce tangible and measurable results which will effectively communicate the value of collections and the expertise of staff. Yet developing a sound understanding of audience needs prior to informing institutional strategies can be key when laying the ground for impact. For digital collections, Dobreva, O’Dwyer, & Konstantelos (2012) argued that involving users in the digitisation process is an approach that can ensure that the resources created are relevant to the needs of different audience groups and, thus, can remain in use for the long term leading to greater impact. Similarly, Mills (2015) discussed how institutions can involve users in processes such as the selection of material for digitisation and the development of digital collections while keeping institutional priorities in mind, and reflected on the value of this approach.
In addition, recent work has highlighted the importance of qualitative information or mixed methodological approaches in understanding the multidimensional impact of cultural work. For instance, it is hard to capture and truly comprehend the effect of scholarship and culture on the ‘personal experience’ through statistical data alone (Crossick, 2017). Regarding libraries and their services, Brophy (2008, p. 16) argued over the importance of context when measuring and assessing performance and noted that ‘[…] because of the complexity of information and communication flows in blended (real and virtual) environments […]’, qualitative methods are more appropriate for gathering evidence that tells ‘rich’ stories of achievement. Some of the reasons why qualitative methodologies are particularly suitable for gathering information related to the significance and impact of unique and distinctive collections become apparent through the Marsh, Punzalan, Leopold, Butler, & Petrozzi (2016) paper. By focusing on digitised ethnographic collections, the authors explained how quantitative data can undermine the importance such collections have for specific communities and, thus, hinder their real value; qualitative methods, such as collecting stories, were deemed more appropriate for understanding and articulating the significance of these collections.
In 2016, RLUK2 launched its Special Collections Programme (SCP)3 which employs a similar approach towards the collection of evidence that showcases the impact of collections. The SCP is a flexible programme of networks, tools and opportunities designed to improve access to, and use of, special collections across RLUK’s membership; it has a special focus on encouraging wide-ranging audience engagement with collections and inclusion of collections in teaching and learning programmes at institutional level and beyond. Moreover, advocating for the significance of special collections and the integral contribution of library staff in academic life is an important aspect of this programme; the RLUK Special Collections Leadership Network (SCLN),4 which is a professional peer network for RLUK members in a senior special collections position, plays an important role in this area.
Activities developed as part of the programme form part of our response to the sixth challenge of the RLUK Strategy ‘Reshaping Scholarship’.5 For example, we collected stories that currently form an online resource showcasing the potential and value of special collections to contribute to institutional impact.6 These case studies and the issues they raised led to the development of the project which is presented here. The aim was to explore the role that special collections and archives play in enabling research libraries to meet their impact goals as well as investigate the ways the impact resulting through relevant services and activities is evidenced. The next sections of this paper will look at the methodology employed as part of this project and its results, including suggestions on how research libraries can strategically work towards increasing the impact of their special collections and archives in the digital age.
In the context of the RLUK Special Collections Programme (SCP) and, thus, in this project, the term ‘impact’ was used to describe:
The demonstrable contribution that the unique and distinctive collections of our [RLUK] members can make to research, teaching and learning as well as to society and its wellbeing.
This definition also takes into account the ‘pathways’ through which impact is achieved, such as the audience-focused strategies developed by most of RLUK members to increase access to and engagement with special collections and archives regardless of their format. Moreover, in the context of this definition, we recognise the unequivocal role that library staff members, including their skills and expertise, play in the implementation and successful delivery of these strategies. Finally, when referring to the ‘value’ and ‘significance’ of special collections in this work, we aim to describe the potential of these collections to lead to positive, short or long term, change within their institutions and beyond.
The twenty-six case studies collected in the context of the RLUK Special Collections Programme (SCP) were categorised based on the themes of ‘places and spaces’, ‘people and projects’, and ‘research outputs’. Each case study comprised one or a combination of the following: a published or previously unpublished narrative (e.g. blog posts, journal articles) where library staff and/or researchers share their experiences within specific projects and reflect on the journey to success; a film or a sound recording where library professionals and/or researchers discuss projects based on special collection and archival material; one or more project outputs outside the aforementioned formats, such as a website, a repository, an exhibition. As mentioned earlier, these provided the initial evidence and inspiration to further investigate institutional practices with regards to the impact achieved through special collections and archives.
More specifically, the objectives of the project were:
- To explore the strategies employed by RLUK member libraries towards achieving and evidencing impact through special collections.
- To identify the opportunities and challenges involved in developing and implementing relevant strategies and activities.
- To identify and share best practice around impact-enhancing strategies for special collections.
- To advocate for the value and significance of special collections and archives.
The research questions we set to answer for the purposes of meeting the above objectives are:
- What pathways (physical and digital) are followed by research libraries in order to increase the value and impact of unique and distinctive collections?
- What are the types of impact achieved through special collections?
- How do research libraries capture and evidence the impact of their special collections?
- What are the characteristics of successful impact stories?
- What are the benefits and challenges of engaging in impact-enhancing activities as well as the lessons learnt?
For the purposes of answering the research questions of this project and, thus, meeting our objectives, we designed a qualitative survey which enabled us to gather more information around the role of special collections and archives in generating impact and the ways this is evidenced across the RLUK membership. The application chosen for the design of the survey and the collection of data was Google Forms. The questionnaire entailed mostly open ended questions (sixteen out of twenty one questions) aiming to capture as much detail as possible regarding the views and experiences of our members on the topic; thus, attention was given in providing participants with the opportunity to describe their own institutional case and share their experiences through including a long answer text box under most of the questions.
The survey was circulated through the RLUK Special Collections Leadership Network (SCLN) and was completed by representatives from sixteen member university libraries (out of thirty seven in total) across the UK and Ireland (see Table 1). One response per institution was received. The amount of in-depth qualitative data received through the survey made this a fair level of response, which combined with the evidence provided through the case studies, gave useful insights into the practices of RLUK institutions.
|Edinburgh University Library|
|Queen’s University Belfast Library|
|University of Leeds Library|
|University of Sussex Library|
|University of Sheffield Library|
|University of Reading Museums and Special Collections|
|King’s College London Library|
|London School of Economics Library|
|Queen Mary University of London Library|
|Cambridge University Library|
|Durham University Library|
|University of Leicester Library|
|The Library of Trinity College Dublin, The University of Dublin|
|University of Glasgow Library|
|SOAS Library, University of London|
The survey respondents were mostly professionals responsible for the management of special collections and archives and their roles ranged from archivists and keepers of archives to managers and heads of special collections, archives and galleries. Amongst the participants were also one director and one assistant director with strategic responsibility for collections. Each survey respondent was given a participant number based on the order they completed the survey (e.g. Participant 01, Participant 02).
Participants were asked to respond to questions on the below areas:
- The types of special collections and archives they hold as well as their audience groups.
- Their institution’s strategies around impact, including any definitions they may have.
- The types of services they provide or activities they design with the aim of increasing the impact of their special collections and archives.
- Any successful initiatives which were based on the use of special collections and archives and were led by their institutions, including the benefits and challenges they entailed.
- The methods they employ for capturing and measuring the impact of collections.
The data was collected and analysed between June and August 2018 while, during the interpretation stage, relevant literature in the field was consulted. The analysis of the data was guided by our research questions which were designed principally with the RLUK strategical goals and the needs of the RLUK community and stakeholders in mind. The case study evidence that had been collected prior to the launch of the survey was also taken into consideration during the interpretation stage. Preliminary results from the analysis were presented at RLUK stakeholder meetings, such as the RLUK Special Collections Leadership Network (SCLN) and RLUK Funders Network meeting; feedback gained during these meetings not only validated, but also enriched our understanding of the initial results.
Before proceeding with the presentation of the results, though, it should be noted that this was an exploratory study, conducted within the framework of the RLUK Special Collections Programme (SCP). The aim was to encourage conversation around issues that RLUK members face and are related to the value and significance of their special collections. Thus, the results presented here may not necessarily reflect the situation in other libraries across the UK and Ireland or beyond. The next sections aim to unpack the results with regards to practices and methods used for capturing and evidencing value, including the opportunities and challenges in the digital age, and discuss potential solutions for improving institutional approaches to impact.
4.1. Audience Engagement and Impact
The survey results revealed the vastness of the holdings of member libraries across the UK and Ireland as well as their historical and institutional significance. These collections and archives include primary resources from across the centuries and on a variety of themes which make them of national and international significance and, as Participant 02 also argued, ‘of lasting research value’. The diversity in the types and formats of materials held by participating institutions ranged from print book and manuscript collections to film archives and collections of cultural artefacts and artworks. Moreover, the majority of respondents stated that their digitised and born-digital holdings were constantly growing while, amongst the examples provided, there were designated collections,7 which had received formal recognition of their value. The importance that these collections have for their institutions was also evident. For example, correlations between the themes represented in the collections and the research areas an institution was famed for were frequently apparent; these constituted indirect, but strong references to the value of these collections for their local academic communities as well as their role in raising the profile of a university.
Regarding the users of these resources (Figure 1), the below graph shows that researchers and students were the core audience groups of all institutions participating in the survey; it should be reminded that the survey participants were representing UK and Irish university libraries.
Apart from the academic and student groups, though, the majority of respondents said that several of their collections were frequently used by members of the general public, including local communities and interest groups. Others reported use of their collections by stakeholders, professional services and other members of staff from their home institution, schools and alumni. Finally, several of the professionals participating in this study highlighted the focus currently placed on extending access to new audiences; as some of them noted, this is often a requirement by funding bodies, such as Heritage Fund,8 supporting initiatives based on cultural heritage collections. Although it may not be immediately apparent, audience development practices as well as factors such as funders’ requirements can play an important role when designing strategies with the aim of achieving impact and maximising the value of collections.
More specifically, many of the participating libraries mentioned adopting the impact definitions of their home institutions, meaning the universities where they were based. These definitions were often developed to either follow the Research Excellence Framework (REF) criteria or to refer to similar goals and values; thus, they placed a strong focus on research and the contribution that excellence in this area can make to society. The role that education and other activities led by the university can play towards improving society’s health and wellbeing was sometimes noted as well.
However, within this context, several of the libraries had also developed their own working impact definitions; these communicated not only the contribution of the library to the broader mission of the university but also their responsibilities outside academia. Impact, from the perspective of the library, was seen as the contribution it can make to research and teaching through collecting or creating quality resources, and developing valued services and partnerships. The well-being of non-academic communities through broadening access to and engagement with library collections was also part of the mission of many institutions.
Given that this dual responsibility of modern research libraries – to produce work that has both academic and civic benefits – is not usually reflected through the university’s impact statements, it was not surprising that many of RLUK members chose to create their own definitions. More specifically, the language used as part of the universities’ impact definitions, with its strong focus on research and the Research Excellence Framework (REF) criteria, cannot always communicate the value and positive effect of the audience-focused strategies and activities led by libraries. As Participant 01 argued, research libraries often use terms such as ‘engagement’ to refer to impact which echoes the language used by other cultural heritage institutions and funders to communicate about similar issues.
A challenge that may come out of employing an impact statement which does not fully serve a library’s strategy is that any relevant activities designed and delivered to meet the goals set by that definition may fail to reveal the full potential of special collections and archives as well as undermine the expertise and contribution of staff. This, in turn, can lead to difficulties in evidencing and measuring the value of collections and related services and hinder reward and recognition for libraries. Therefore, it is worth noting that one RLUK member (Participant 11) reported employing a definition which was closer to the way funding bodies, such as the Arts & Humanities Research Council (AHRC),9 describe impact. Another (Participant 16) underlined the importance of the RLUK Special Collections Leadership Network (SCLN) in influencing the way they shape their definition and approach related to impact. Finally, it should be mentioned that there were two cases where no institutional definition of impact existed or could be found; activities around capturing and measuring the impact of library collections and services were based on personal initiative or were conducted on an ad-hoc basis.
4.2. Striving for Impact: Opportunities and Challenges
The majority of participating institutions were regularly engaged in activities that aimed to increase the impact of their unique and distinctive collections (Figure 2).
Some of the most frequently mentioned activities were:
- The creation of new buildings and spaces with state-of-the-art facilities (e.g. labs, reading rooms)
- Collection development activities to ensure that library resources remain relevant and accessible
- Exhibitions (physical and digital) and other public engagement programmes and events (e.g. festivals, open days, public talks, tours)
- Loans of collection items to external exhibitions
- Collaboration in research and other cross-sector projects and initiatives
- Participation in cross-departmental schemes and programmes (e.g. new institutes)
- Involvement in teaching activities, such as designing and delivering course modules
- Co-authoring/participating in funding proposals within/outside academia
- Digitisation and digital resource development initiatives
- Participation in advisory group panels and other networks
- Contribution to the development of Research Excellence Framework (REF) case studies based on special collections
- Hosting research fellowship, internship and visiting scholars programmes
- Developing communication strategies and using social and traditional media
According to the participating libraries, the prevalent reason for getting involved in activities such as the above was the benefits that could be gained for the library and, by extension, its home institution. These activities were commonly perceived as opportunities to showcase the richness and value of special collections and archives as well as the work conducted in the library to a broad audience of existing and future users and stakeholders.
In more detail, the reported benefits of leading or participating in impact-enhancing activities were:
- Demonstrating the range and diversity of the library’s unique and distinctive collections and raising awareness about their potential.
- Facilitating re-interpretation and innovative use of collections as well as creating new learning opportunities.
- Contributing to the mission of the university, participating in its governance structures in a positive way and raising its profile.
- Contributing towards the improvement of the local economy (e.g. through hosting exhibitions and attracting visitors).
- Building a portfolio of activities that could lead to further funding, reward and recognition at local and national level.
Sector-wide developments and national initiatives, such as celebrating anniversaries and launching new funding schemes, often provided suitable ground for engaging in a range of activities. Furthermore, based on the survey results, it was apparent that institutions were taking advantage of the opportunities that the digital age offers for outreach and advocacy through organising and delivering relevant initiatives – from developing online and social media presence for specific projects to organising digital exhibitions and campaigns – that enhanced the impact of special collections and archives. As can be seen in Figure 3, though, digitally-oriented activities conducted for the purposes of achieving greater impact can also have additional benefits for institutions as they could be linked to or trigger other initiatives that highlight or facilitate the library’s and its staff members’ work around collections.
However, although respondents agreed that the overall experience of being actively involved in such initiatives has been positive, it has not been without its challenges. Some of the most frequently encountered difficulties were related to the logistics of organising, running or participating in certain types of activities, such as exhibitions or collaborative projects (e.g. inconsistency of funding, licencing issues, managing staff time and availability).
For those involved in larger scale projects, negotiating the conflicting priorities of many stakeholders could be problematic; this happened particularly in the cases where the library was not seen as a full-fledged partner. Others reported facing challenges when designing initiatives to engage new audience groups; trying to sustain momentum for long term projects; and dealing with the media. Lastly, on top of the above difficulties, activities developed and delivered in the digital environment could present extra challenges for libraries and their staff members. This could happen particularly in the context of initiatives where new technologies were introduced and it was necessary for staff members to learn how these could be best used not only for the purposes of the project, but to maximise impact as well.
4.3. Capturing and Evidencing the Value of Collections in the Digital Age
Regarding the types of evidence gathered by member libraries with the purpose of showcasing the impact of special collections and that of related activities, they included:
- Anecdotal feedback
- Exhibition visitor data and demographic data
- Ticket sales
- Data from interactive exhibits
- Results from traditional questionnaires and feedback/survey forms (e.g. teaching)
- Data from interviews with users (e.g. researchers and students)
- Item or service request numbers
- Metrics and automated statistic gathering from digital initiatives
- Evidence of activity provided by academic ‘champions’, responsible for advocating for library collections
- Media coverage and high social media engagement
- Examples of fostering future research or follow-on projects
- Citations in publications
- Occupancy rates for reading and search rooms; items produced for researchers and classes
- Data produced for evaluation and assessment reports
Thus, according to our findings, initiatives based on special collections and archives were deemed successful if they had achieved several of the following:
- Received positive feedback; generated publicity
- Attracted a high number of visitors
- Led to increased use of the collections by existing and new users
- Received strong endorsement from senior levels in the university
- Received the stated willingness of lenders to be involved in future projects, such as exhibitions
- Had a high number of hits to a project’s or other resource’s website
As a result of success, collection professionals were often approached by people from within their home institution, other organisations or the general public who expressed an interest in an initiative or other aspects of the library’s work; for example, there would be increased interest by local academics in collaborating with the library (e.g. to co-curate an exhibition or to co-author a funding application).
As part of developing a sound understanding of how research libraries capture evidence that demonstrate the value of their collections and that of related services and activities, we asked RLUK members to elaborate on how they learn about institutional and third-party projects (especially research projects) that utilise parts of their collections or reference any contribution that library staff members have made as well as how they capture outputs from such projects. Generally speaking, it became apparent that RLUK member libraries had a good level of awareness of how collections were used as part of institutional projects. This was true especially in the cases where a library was directly involved in a research project, had provided advice on a funding application or one of its collections was referenced in a Research Excellence Framework (REF) case study.
Moreover, working closely with local academics and other users of the collections (e.g. artists) as well as developing relationships with key people across the university (e.g. PIs, research assistants) were considered as effective methods for learning about projects and related outputs that could be used as examples to showcase the value of collections. Others noted that being part of networks and advisory group panels, collaborating more closely with other library teams (e.g. research data team) or receiving direct requests for access, supply of digital surrogates, or supply of additional information relating to collection/items had resulted in discovering about such projects and outputs.
Yet, the majority reported that it was harder to find out about projects outside the university that utilised parts of their collections (physical or digital) and capture their outputs. Thus, the impact that library resources or the work of staff had on these projects could not be evidenced. Capturing research outputs of third-party projects was often an ad-hoc activity or depended on personal initiative. Searching for citations through Google Books or through social networking websites such as Academia10 and ResearchGate11 was common practice but, as Participant 02 argued, many found this method cumbersome and ineffective.
‘Other methods of capturing impact might be through mentions in academic publications or through involvement with resources like Academia.edu or ResearchGate or searching Google Books for mentions though this is a cumbersome and rather blunt method.’ (Participant 02)
Projects involving use of printed sources were particularly hard to locate as citations of individual copies were rarely made in resulting publications. Whenever member libraries were made aware of the existence of such projects, they requested to be sent copies of books or be alerted to publications and other resources involving use of their collections, but this often occurred when an item request (e.g. for an image) was involved.
Other reported difficulties were specifically related to the evaluation of digital resources as well as the measurement of long term impact of the programmes and activities led by the library as several members highlighted the lack of structured methods for capturing related evidence. For instance, Participant 01 noted the difficulty in capturing the impact of digital resources while Participant 16 commented on the challenge their institution faces with regards to tracking the long term impact of activities, such as events:
‘[…] There are no real systems in place to capture what people are doing with our collections. In the digital environment, we have even less information.’ (Participant 01)
‘It is harder to measure the longer term impact of the event – while it was well received by those who attended (which is a valid outcome in itself) we do not have an easy way of identifying any increased use of our service that may have followed.’ (Participant 16)
As part of investigating potential methods for tackling challenges such as the ones mentioned above, some RLUK members said that they were using or planning to use openly available toolkits provided by funding bodies or published independently, such as the ‘Making Digital Work’ toolkit by Arts Council England (ACE),12 UKRI’s logic model approach to planning and evaluation,13 and Tanner’s (2012b) ‘Balanced Value Impact Model’.14 For institutions using or planning to use such a toolkit to assess the value of their physical or digital collections, it was important that these were recognised and accepted by funders and other stakeholders. Moreover, the chances of employing them were greater if others in the community were familiar with them or had used them in the past; the RLUK Special Collections Leadership Network (SCLN) has often been a forum for discussing relevant issues. This may explain, to some extent, why other models and frameworks, such as the international standard ISO 16439:2014,15 employed often by libraries in Europe were not mentioned (e.g. see Hall, Cruickshank, & Ryan, 2019; Poll, Boekhorst, & Mundt, 2007). Yet, it should be noted that several of the methods for assessing impact listed in the international standard ISO 16439:2014 (e.g. see Creaser, 2018) were regularly found to be used by the institutions participating in this study (e.g. statistics, surveys, interviews). There were also few examples of assessment programmes developed and led by libraries, such as the ‘Futurelib Innovation Programme’16 by the University of Cambridge which, as Participant 11 argued, can be used for evaluating public programmes. Finally, it should be noted that few institutions did not have any structured ways of measuring the impact of their work.
Therefore, to improve the process of measuring and capturing the impact of special collections and archives in the digital age, our results showed that a number of actions need to take place which are presented in Table 2.
|Measuring and capturing impact: suggestions for improvement|
|Take advantage of new tools and approaches to measure reach and impact electronically - collaboration with other teams within the library can be key (e.g. IT).||Develop better methods for capturing impact through mentions in academic publications, especially from external projects.||Develop a set of measures and benchmarks that will offer a more standardised way to measure impact and help bring funding back to the library. For instance, a standardised citation format for special collections would be useful in this instance as citations could be more easily measured.||Consider different approaches for evidencing the two main types of impact for research libraries, academic and civic, as well as capturing long-term impact of library activities based on special collections and archives.|
As it became apparent through our research questions and objectives presented earlier, this research was of an exploratory nature; yet, the type of collected data enabled the development of the above recommendations. These were greatly based on the answers received through the survey and formulated, principally, with RLUK members in mind. Given that relevant literature was consulted during the interpretation of the data, including previous RLUK work in the area, this was also taken into account when appropriate. The thinking behind the recommendations in Table 2 is unpacked below:
Recommendation 1: Take advantage of new tools and approaches to measure reach and impact electronically – collaboration with other teams within the library can be key (e.g. IT).
Digitisation and the acquisition of born-digital material have led to a wealth of digital resources, increasing interactions in the digital environment. Gathering different types of evidence of use of digital and online resources, in addition to that of physical material, is crucial for providing a more holistic view of the reach and impact of library collections. Yet, as technology progresses, it is important to stay up to date with regards to latest tools and methods for capturing and evidencing value and impact. In the face of challenges mentioned frequently by our participants (as mentioned earlier), such as limited time and lack of resources or skills, collaboration between different teams in the library is important. Combining the expertise of different people will increase the ability of the library to take advantage of the most current technological and methodological innovations.
Recommendation 2: Develop better methods for capturing impact through mentions in academic publications, especially from external projects.
Making an impact on research is one of the strategic priorities for most research libraries; tracking the impact of collections through academic citations can be an effective way to evidence value. Yet, even though tracking citations in publications made by an institution’s local academic community was achievable for the majority of participants, current popular practices (e.g. through Google Books or social networks) for identifying citations in publications by external audiences (e.g. international academics) were not yielding the desirable results (as also stated by Participant 02). More systematic and comprehensive methods and approaches for capturing academic citations are needed to make the process not only efficient, but also simple to carry out.
Recommendation 3: Develop a set of measures and benchmarks that will offer a more standardised way to measure impact and help bring funding back to the library. For instance, a standardised citation format for special collections would be useful in this instance as citations could be more easily measured.
Opening up special collections and archives was a key part of the impact agenda of the research libraries participating in this study. Demonstrating the important role that physical and digital special collections and archives can play in research, teaching, and public engagement is vital for securing internal or external funding and other resources for the library. Measures and benchmarks that facilitate the gathering of relevant evidence in a more standardised way will be useful tools towards achieving this goal. Yet, these will need to be flexible enough to be applicable to different situations; for example, they will need to take into account that institutions need to collect evidence from interactions with collections in both the digital and physical environment.
Given that lack of skills, particularly digital, was frequently reported to be a hindrance in capturing impact effectively, relevant training should be introduced alongside a set of measures and benchmarks to encourage adoption. Also, making sure that the standards are approved by different stakeholders in the community, such as libraries, funders, or specific audience groups (e.g. researchers) will increase credibility as well as raise awareness and, thus, long-term use. The need for a standardised citation format for special collections was an issue raised by participants in this study; this need, which has not been met yet, was also highlighted through another study commissioned by RLUK, The National Archives and Jisc (The Research Base, 2018). Developing a model that will promote accurate and consistent citations to special collections and archives repositories, for example in academic publications, will make the measurement of impact an easier task.
Recommendation 4: Consider different approaches for evidencing the two main types of impact for research libraries, academic and civic, as well as capturing long-term impact of library activities based on special collections and archives.
Many institutions in this study did not have a consistent way of tracking and evidencing the value of their collections and services, while achieving this through digital means could be even more challenging. Ensuring that a variety of tools and methods (quantitative, qualitative, or mixed) is employed to capture and measure this evidence can significantly increase the ability of institutions to evidence both short-term and long-term impact, including the two main types for research libraries, academic and civic. Yet, a strategic plan outlining a methodological approach towards capturing and measuring relevant evidence should also pay attention to the language used to describe and communicate issues around impact and value.
Research libraries in this project were often found to create their own definitions of impact as terminology used by universities, closer to the Research Excellence Framework (REF) criteria, could not fully reflect the breadth of activities led by libraries and their value for their communities. This paper argues that there is a need to develop a common language to describe the value and significance of collections that better serves the strategic goals of research libraries and which is recognised by funders and other stakeholders. This argument is based on our findings but also takes also into account discussions held within the RLUK Funders Network where preliminary results of this study were presented. This approach will ensure that the methods used to evidence and capture the impact of special collections are recognised by home institutions and beyond and, thus, the rewards for the library are more easily yielded.
The above points, along with the study’s other results, were used as the basis for generating discussion within RLUK networks; for example, by comparing this study to previous RLUK work in the area (e.g. Cullingford et al., 2014) to understand the progress RLUK members have made with regards to impact. Finally, the report informed the development of activities as part of the consortium’s strategic plan on special collections and was used for advocacy purposes and engagement with RLUK stakeholders.
Based on our results, it was clear that RLUK members are frequently engaged in activities that aim to increase the impact of their collections and services and are highly interested in methods that enable them to capture and showcase their value and potential. It is worth noting that, compared to the findings of the RLUK report by Cullingford et al. (2014), which addressed similar issues, RLUK research libraries are now increasingly taking advantage of technology and fundraising opportunities to develop activities that raise the profile of their institutions and collections. More specifically, while in the 2014 RLUK report a great part of the discussion focused on how special collections can contribute to their institutions (Cullingford et al., 2014, pp. 17–30), this project showed that research libraries have been already contributing towards institutional goals through their collections and a wide range of activities and collaborations based on them.
The recent shift towards audience-focused strategies and the steps taken by members to increase physical and digital access to collections and engage with a variety of audience groups have offered greater potential for producing measurable outcomes that showcase the value and impact of collections. In that respect, our findings confirm previous studies’ arguments (e.g. Carter, 2012; Dupont & Yakel, 2013) on the potential of audience-led strategies to generate better opportunities for demonstrating the significant role that special collections and archives can play in enabling institutions to meet their impact goals.
However, in order to be able to develop effective success strategies and bring benefits back to the library, some additional factors should be taken into account. For instance, at the university strategic level, there should be a good understanding of the role that the library can play in creating impact. The opportunities offered by the digital age to open up collections, such as through increasing access to uncatalogued material still existing within institutions as well as improving discoverability of digital collections will greatly facilitate this purpose and will enable the collection of data that showcases value. Advocating for the value of collections within the institution and beyond and developing relationships with senior members of the university are considered crucial steps. According to our findings, forming cross-departmental collaborations was not only an effective way of achieving impact, but also a great way to communicate the potential of collections and gain the support of home institutions (also see Recommendations 1 in section 4.3).
Apart from the above points, given that the terminology used by universities to describe impact – closer to the Research Excellence Framework (REF) criteria – does not always serve the strategic goals of the library, it is worth collectively considering the development of a common language to describe the value and significance of collections and advocating for its acknowledgment by university and other stakeholders, such as funders (also see Recommendation 4 in section 4.3). This approach will ensure that the methods used to evidence and capture the impact of special collections are recognised by home institutions and beyond and, thus, the rewards for the library are more easily yielded. As part of this approach, the way libraries communicate about their unique and distinctive collections and their impact to their audiences should be also re-considered; looking at relevant examples from the broader cultural heritage sector (e.g. museums) may provide some useful insights.
5.1. Professional and Skills Development for the Digital Age
In this paper, aspects of the evolving role of the collection professional became particularly evident. As part of being engaged in impact-enhancing activities, staff members responsible for the management of special collections and archives were found to collaborate with other library teams, stakeholders and audience members, such as researchers, and undertake responsibilities that expanded their traditional role. For example, getting involved in teaching, research or public engagement activities was common practice for most of the participants in this project. Furthermore, as part of these new responsibilities, collaboration and communication skills were considered necessary for engaging with a variety of stakeholders while good digital skills were often needed for working in digital projects and capturing and communicating the impact of collections and related services.
Compared to the 2014 RLUK report (Cullingford et al., 2014, pp. 21, 31), there was an emphasis on the need to work more collaboratively with others within the institution and beyond and suggestions were made on how the skill set of collection professionals could be put to good use (e.g. in teaching). Based on our findings, we can confirm that steps towards this direction have been taken by the majority of RLUK institutions, while our results demonstrated that the image of special collection departments has been continuously transforming – from ‘closed’ spaces to hubs of creativity and innovation – and, hence, are in agreement with those of recent studies on the topic, such as Tam’s (2017).
Yet, as part of supporting the changing role of collection professionals, institutions need to invest in skills and professional development for staff. Supporting staff in developing their digital skills and confidence in using new technologies for the purposes of increasing, capturing and measuring the impact of collections was an issue that came up in this project. This will need to be addressed in order to fully take advantage of the opportunities offered by the digital age (also see Recommendation 3 in section 4.3). Accordingly, ensuring that job descriptions effectively communicate the variety of activities potential employees will need to be engaged in as well as the skills they should have is necessary. In addition, developing activities that are closely tied to the library’s goal will ensure that the time invested by staff members can be more easily justified, whereas institutional support will also prove valuable in this area. Finally, given the active participation of collection professionals in collaborative research projects, advocating for the inclusion of library staff members in funding applications as equal research partners will help towards bringing more recognition and reward to the library.
5.2. Methodologies for Impact in the Digital Environment
Previous work in the area has shown that cultural institutions, including research libraries, have been finding it challenging to collect evidence in a way that produces meaningful and trustworthy information around the use of collections (e.g. see Cullingford et al., 2014; Hughes, 2012). According to the results of this project, most of participating RLUK members showed a good level of awareness with regards to how their collections were used within their institutions which made it easier for them to capture and showcase value.
On the other hand, the majority of institutions reported difficulties in tracking and capturing the impact of collections when used for external projects, measuring long term impact of library resources and services or effectively evaluating the use of their digital resources. Capturing and demonstrating the value of digital collections and resources for scholarship, teaching and public engagement is an activity of particular importance for institutions in the digital age as they are increasingly called by funders and other stakeholders to justify the resources spent on digitisation and digital resource development. According to Showers (2012, p. 71), employing a more strategic approach to impact, including taking into consideration issues around impact throughout a project’s lifecycle and developing an understanding of audience needs before the digital resource creation commences, can lead to a more structured and effective resource evaluation process.
Moreover, given the lack of structured methodologies for capturing types of evidence such as research outputs effectively, a collective response towards this challenge may be the way forward (e.g. collectively developing standardised ways of tracking and measuring value) (also see Recommendations 2 and 3 in section 4.3). A recent publication commissioned by RLUK, The National Archives and Jisc and produced by The Research Base (2018), exploring ways to standardise references to unique and distinct collections (UDC’s) held in repositories across the UK, is an important step towards developing a better understanding of the impact that special collections and archives have on academic research and beyond. Additionally, research libraries may benefit from looking at the broader cultural heritage sector for relevant models and new ways of capturing and showcasing impact as organisations, such as museums, have a longer tradition of engaging in similar activities.
Regarding methodologies, it is also worth considering how quantitative and qualitative methods can be best combined to capture the different types of impact that concern libraries, academic and civic as well as the long term effect of libraries’ activities and resources (also see Recommendation 4 in section 4.3). Employing the right approach will enable institutions to better showcase the broader value of collections and related services to stakeholders, such as cultural heritage funders, who may be interested in evidence that proves the impact of collections on a variety of areas, and not only on research. Additionally, qualitative methods are generally considered more appropriate for capturing and fully illustrating the effects of long term impact, such as change in practices and behaviour. A suitable methodological approach towards capturing and measuring relevant evidence should also take into account how language is used to refer to impact and value and ensure that this meets the strategic goals of the library. Finally, advocating for a change in how collections are referenced in research funding applications and publications will prove beneficial when it comes to tracking down significance and impact (e.g. tracking follow-on projects).
The purpose of this paper was to further our understanding of the role that special collections and archives play in enabling institutions to meet their impact goals as well as investigate the ways that impact resulting from relevant services and activities is evidenced. Our findings showed that RLUK members frequently lead or participate in activities that aim to increase the impact of their collections and services and are highly interested in methods that enable them to capture and showcase their value and potential. A shift towards more audience-focused strategies is increasing collection professionals’ involvement in research, teaching and cross-institutional collaborations as well as public engagement activities while opening up special collections and archives and transforming their image.
Yet, as the shift towards openness is not even across institutions, there is still work to be done with regards to maximising opportunities for impact in the digital age; improving discovery of conventional – especially uncatalogued – and digitised material is necessary in order to enhance the value of collections. Moreover, this paper makes four recommendations for improving processes around capturing and evidencing the impact of collection-based activities and services in the digital age. More specifically, certain challenges, such as the need for more structured methodologies to capture the long term impact of collections or to better understand how collections and digital resources are used outside institutions, will need to be addressed. Taking advantage of new tools and approaches to impact will strengthen the library’s potential to respond to issues associated with the constant technological change. Investing in skills development for staff, especially digital, will support collection professionals in their continuously evolving roles. Ultimately, this will also support the library’s aim to maximise the reach and impact of its unique and distinctive collections. Developing collectively more standardised ways to measure impact and value that are recognised by stakeholders will increase the credibility of evidence. It will also enable institutions to build portfolios of successful impact cases that can be used for purposes, such as advocacy and lobbying, that aim to bring more funding and recognition to the library.
Finally, being aware and using a mixture of methods, qualitative or quantitative, is an important element of an efficient approach to impact. This approach should also take into account the choice of language used to describe and communicate about related activities. At the moment, the unsuitable terminology existing in many definitions of impact that do not serve the goals of research libraries fail to reveal the potential of collections and the importance of the contribution of staff members. The development of a common language around impact which is acknowledged by home institutions and other collection stakeholders can significantly facilitate the capturing and evidencing of value.