Evidence-based practice is an approach that can support libraries in demonstrating value and impact to stakeholders. It is a structured way of working that brings rigour to the processes of collecting, interpreting and applying valid and reliable evidence to support decision-making and continuous service improvement in librarianship (Howlett & Thorpe, 2018). This case study describes the development of a culture of evidence-based practice at a multi-campus, regional university library in Australia. Through organisational change, dedicated champions and defined expectations of staff, University of Southern Queensland (USQ) Library is building capacity in evidence-based practice to demonstrate value to stakeholders, to gain a deeper understanding of clients’ needs and experiences, to promote robust decision-making and to improve service delivery.
2. Theoretical framework
Evidence-based practice in librarianship first appeared as a term in the late 1990s (Eldredge, 2000) building on the experiences of health librarians who were supporting clinicians practising evidence-based medicine. Many other professions have adopted and embraced evidence-based approaches, including management, executive coaching, career development, public policy and education (Miller, Partridge, Bruce, Yates, & Howlett, 2017). Evidence-based practice is not just about defining measures and undertaking data analysis, nor does it end with gathering and reporting data. Rather, evidence-based practice promotes applied approach that is ongoing and reflective, in which library staff position themselves to respond to challenges and leverage opportunities within their library’s local context (Thorpe, 2018). Measures are the means through which service is improved and professional practice is continuously developed (Howlett & Thorpe, 2018). It is ongoing, iterative and reflective in nature. Koufogiannakis & Brettle (2016) argue that evidence-based practice is more than just “doing”; it is a way of being. By being evidenced-based, librarians can question their practice, gather or generate evidence and use evidence to make informed decisions about value and impact. Within academic research environments, library staff need to demonstrate value to stakeholders and decision makers within the university with the power or influence to support projects, fund initiatives and prioritise capital works. O’Sullivan and Partridge (2016) suggest that much of this work happens outside of formal reporting to the university’s executive, often by library staff who may not have a deep understanding of the value proposition of the library. Rather, it happens via the myriad of interactions library staff at all levels have with students, staff and community members of all kinds (O’Sullivan & Partridge, 2016).
3. An applied approach to evidence-based practice in academic libraries
University of Southern Queensland (USQ) is a regional university in south-east Queensland, Australia, with more than 27,000 undergraduate students (University of Southern Queensland, 2019). More than 75% of the student cohort study online or via distance. USQ has been a higher education institution for over fifty years and is known for a strong social justice focus as the largest provider of higher education to Australian incarcerated students (Farley & Hopkins, 2018). USQ Library supports students on three campuses, those studying online across Australia and the world, and students studying in offline mode in Australian correctional centres. In 2016, the Library underwent a transformational change process culminating in a new organisational structure (O’Sullivan & Partridge, 2016). The four principles underpinning the organisational renewal were:
- To engage with clients in new and different ways around the future of the Library
- To enable library staff to identify trends that may impact services into the future
- To explore alternative futures and devise strategic responses to the identified trends
- To establish an evidence base to inform and guide workforce and strategic planning (O’Sullivan & Partridge, 2016).
This approach reflected the broader context of Australian academic libraries, with the Council of Australian University Librarians identifying a long-term aspirational goal to nurture a culture of evidence-based thinking and communication across the nation (Owen, Peasley & Paton, 2017). As part of the transformational change, the Director, Library Services, articulated the expectation of evidence-based practice as relevant in all staff roles, by including in every position description a statement that staff will participate in a culture of evidence-based practice, use scholarship to inform and innovate their practice and, at higher levels, contribute scholarly work to the higher education and library and information science domains. A dedicated role was created to champion and build skills and capacity across the Library and to focus on enhancing the Library’s capacity to innovate and deliver services in an informed and effective way. The role was designed with a mandate that went beyond the scope and capabilities of a data analyst who might focus only on data collection and reporting. Rather the Coordinator (Evidence-Based Practice) has a mandate to work with library staff to develop their skills as evidence-based practitioners and to communicate the value and impact of the library to stakeholders (Howlett & Thorpe, 2018). The incumbent is charged with engaging with library staff to understand their business, identifying the evidence needed to support business improvement for the Library, and advising and supporting staff to engage in evidence-based practice.
Figure 1 visualises how we describe and conceptualise the Coordinator’s position within USQ Library (Howlett, 2018a). It is conceived of and viewed as an enabling role. Evidence-based practice is acknowledged as an iterative or cyclical process when applied in the academic library context through a series of interrelated activities. As Library staff interpret, apply, measure and communicate evidence, our services become better aligned with our strategy and goals and our outcomes and impact lead to greater influence and advocacy on behalf of our clients. Evidence underpins all our work and our staff are central to our success.
Much of the work of the Coordinator role is in building relationships, building staff expertise and capacity, fostering awareness and communicating the benefits of evidence-based practice to teams. The role enables, encourages and empowers USQ Library staff to:
- Access and use the best available evidence
- Interpret and apply evidence to practice
- Evaluate and measure outcomes to build and maintain a robust local evidence base
- Be evidence-based as routine.
Applied and practical examples of evidence-based practice at USQ Library have included:
- The development of a Selection and Evaluation of Electronic Content tool. An evidence-based decision-making tool for the purchase and decommissioning of electronic content (Hay & Pearse, 2018). The evidence generated by the adoption of this evidence-based tool has been successfully used to both identify opportunities for increasing usage of content and to identify cost savings through cancellation of subscriptions that are no longer relevant, affordable or useful for the university’s teaching and research priorities. The tool generates more robust evidence to not only making collection decisions, but also communicate these decisions to stakeholders. It provides the right evidence, based on rigorous data, for decision-making.
- The increasing use of low cost, user experience (UX) techniques in designing the refurbishments of library spaces that are student-centred, sustainable and fit-for-purpose (Howlett, 2018b). An activity to choose furniture for the Springfield campus library had the added benefit of changing the way students viewed the Library. The Library was subsequently described as ‘caring’ by students who participated in this activity. UX techniques provide an opportunity to communicate and collaborate with the student community in improving their university experience.
- The adoption of evidence-based approaches to decisions, such as library opening hours, using analysis of multiple data sources to inform options and solutions. This has included better use and understanding of datasets available from Library information systems, including usage and process analytics. Using data visualisations and reporting tools, staff can ‘see’ the evidence and understand how evidence-based practice is relevant for their work.
- The use of qualitative and quantitative methods to generate evidence to support funding applications, such as the Student Amenities Fees grants. The use of evidence has led to a higher success rate in annual funding applications.
Through a people-centred, evidence-based approach that encourages experimentation, we believe we have fostered a culture of problem-solving and collaboration. In a 2018 university-wide survey of staff satisfaction, 83% of Library staff agreed with the statement: “My co-workers take the initiative in solving problems”, an eight point increase on the previous survey from 2016.
4. A conceptual approach to evidence-based practice in academic libraries
One of the challenges faced in implementing the Coordinator (Evidence-Based Practice) role has been to identify the evidence that demonstrates how this role is making a contribution to the performance of USQ Library. How does an academic library as an organisation become evidence-based? How might an academic library holistically grow in its maturity as an evidence-based organisation? Existing literature has focused on understanding evidence-based practice within the individual experiences of library and information science practitioners (Gillespie, 2014; Gillespie, Partridge, Bruce, & Howlett, 2016; Howlett & Howard, 2015; Koufogiannakis, 2013; Luo, 2018; Miller et al., 2017; Partridge, Edwards, & Thorpe, 2010). While the individual experience is important, a culture of continuous improvement requires supportive organisational dynamics and workplace contexts — a whole-of-library approach to evidence-based practice which guides and develops library services and collections (Gillespie et al., 2016; Koufogiannakis, 2013). Within the academic context, the value of university and research libraries is increasingly judged by its demonstrated ability to help students learn and researchers research.
To explore evidence-based practice at the organisational library level, the Coordinator (Evidence-Based Practice) and Associate Director (Library Experience) interviewed 16 professional library staff from 10 different university libraries across Australia and New Zealand. A qualitative approach with semi-structured interviews was used to develop an initial understanding of evidence-based practice at the whole-of-library level (Creswell, 2012). Participant recruitment used a convenience and purposive sampling approach, drawing on our professional networks to recruit participants who were available during the data collection period. The aim was to capture perspectives from across a range of library roles and functions, to encompass a whole of library experience. Full ethics clearance was obtained from the University of Southern Queensland’s Ethics Committee.
Fourteen women and two men were interviewed. Most participants were employed at Australian universities with one person working at a New Zealand institution. Participants were employed as:
- Liaison/Subject/Outreach Librarians (5)
- Team Leaders/Managers (5)
- Associate Directors/Deputy University Librarians (4)
- Directors/University Librarians (2)
Interviewees reflected on their own experiences with evidence-based practice and the level of evidence-based practice maturity demonstrated by their library. Interviewees were also asked to describe the characteristics of an evidence-based library. Appendix A provides a list of sample interview questions that guided discussion. Interviews recordings were transcribed verbatim and analysed using a thematic, iterative approach (Braun & Clarke, 2006). The data analysis provided description about what evidence-based practice might look like at a whole-of-library level. Through the development and refining of codes the findings evolved into three dimensions of experience, mapped across five levels of organisational maturity.
Together with existing research, knowledge and frameworks, thematic analysis of the interview transcripts identified three focus areas or dimensions through which an academic library can demonstrate evidence-based practice. These dimensions, Process, Engagement and Evidence, have their foundations in existing evidence-based library and information practice frameworks. The dimensions were mapped to five levels of maturity, ranging from least mature (Ad Hoc/Sporadic) through to most mature (Transforming). Figure 2 outlines the five tiers of maturity in the Evidence Based Library and Information Practice Capability Maturity Model (Thorpe & Howlett, 2020). This paper will focus on exploring the dimensions of experience in detail. The tiers of maturity are discussed in an earlier publication (Thorpe & Howlett, 2020).
The Process dimension articulates how evidence-based practice was applied within university libraries. Experiences ranged from not at all, as ‘one off’ or semi-regular basis, to integrated into day-to-day ways of working. Evidence-based practice processes were observed as either reactive or proactive, meaning that the purpose of gathering and applying evidence was not always clearly defined from the onset. Some participants described data collection methods in detail. Others focused on the process associated with making evidence usable or communicating it effectively to appropriate audiences. Experiences of, and exposure to evidence-based practice within their libraries reflected a range of staff skills in assessing and applying evidence to particular needs or practice questions. How the evidence-based practice process manifests correlated with the model proposed by Koufogiannakis & Brettle (2016), sometimes referred to as the 5As model. As well as the five elements from this model – Articulate, Assemble, Assess, Agree and Adapt – the importance of communicating evidence to inform or influence decision making was highlighted as a key element of the Process dimension. At lower levels of maturity within the Process dimension, evidence may be collected but is not used to inform decision making. At higher levels of maturity, gathering and using evidence is a strategic priority and underpins how the library operates and makes decisions. Table 1 demonstrates how the Process dimension was experienced by interviewees at the varying maturity levels.
|Tier 1: Ad hoc/ Sporadic||I’m not convinced at this stage that in a lot of cases it’s the right data or the right evidence. A lot of it seems to just be traffic. We’re counting traffic. We’re not actually looking at why there is traffic. (Interview 16)|
|Tier 2: Justifying||Quite often we are looking for the evidence to support a decision that we already have a gut feel about. (Interview 2)|
|Tier 3: Emerging||Every project, you’ve got to make sure that you have a sense of what is happening, and possible solutions, and how things have been applied and how you might apply them. (Interview 7)|
|Tier 4: Experimenting||Having a really good understanding of what our client needs are actually and then putting those together to really evaluate, test any sort of decisions with your client base to confirm that they are the right decisions that you may need to be making. (Interview 10)|
|Tier 5: Transforming||(Evidence) can help you influence. It can help you make decisions. It can help give you a more sophisticated understanding of things. It can give you a different relationship, especially with academia. (Interview 13)|
The Engagement dimension describes how the library as a whole engages with evidence-based practice: how staff and leaders promote, prioritize and enable evidence-based practice approaches, capacity and capability. Engaging in evidence-based practice included the extent to which library staff were supported to develop their capabilities and skills; whether leaders explicitly prioritized evidence-based practice as a way of working, and whether there was a shared understanding of how evidence-based practice approaches were used to improve services and practice. The role of leadership as a driver of evidence-based practice was highlighted by interviewees and aligned with the five categories of experience proposed by Partridge et al. (2010). The level to which academic library leaders championed, sponsored or modelled evidence-based practice strongly influenced the extent to which the organisation engaged with evidence-based practice. Table 2 demonstrates how the Engagement dimension was experienced by interviewees at the varying maturity levels.
|Tier 1: Ad hoc/Sporadic||It was so informal…it says “We’re just making decisions on an ad hoc basis at this stage.” (Interview 9)|
|Tier 2: Justifying||They’re using systems to actually collate and store data basically. The bit where it falls down is it’s actually probably not being analysed really well. (Interview 16)|
|Tier 3: Emerging||If you want your team to be evidence-based in practice, then you have to allow a bit more time in your projects so they can do what they need to do to make sure that they’ve gathered evidence. (Interview 7)|
|Tier 4: Experimenting||I think it’s going to be about also the leadership expecting – this does happen, but just saying, “Okay, I’m not going to make a decision on that until you give me this data.” It does happen. (Interview 14)|
|Tier 5: Transforming||I see it in terms of the library conducting itself, and being managed, and being driven forward, on the basis of evidence-based practice….something approximating a kind of an ethos of the way people practice things and the way the library is managed. (Interview 11)|
The third dimension which emerged from the interviews examined which evidence sources were used within academic libraries to make decisions and improve practice. This included the types of evidence involved in a library’s process, how evidence is identified and perceived, and an awareness of the limitations of evidence to different situations and contexts. Though closely linked to Process, this element describes an awareness of, and capability to identify, gather and apply appropriate evidence to practice and service delivery. The types of evidence identified by participants indicated those most valued in making decisions within their libraries. Examples ranged from research literature, benchmarking studies and environmental scans to systems-generated statistics, focus groups and qualitative feedback. The interview data demonstrated a variance in the ability of librarians at all levels to use different types and combinations of evidence. The variety of evidence sources mentioned in the interviews correlated with the three categories of evidence defined by Koufogiannakis & Brettle (2016), which is local evidence, research evidence and professional knowledge. The ability to apply evidence from multiple and various sources to the specific local context was highly valued by participants. Table 3 demonstrates how the Evidence dimension was experienced by interviewees at the varying maturity levels.
|Tier 1: Ad hoc/ Sporadic||I can see when I first came on board a lot of decisions were just made on people’s personal opinion, and that still happens to quite an extent, actually. (Interview 10)|
|Tier 2: Justifying||Narrowing down our key performance indicators to a more realistic set of things that are actually performance indicators rather than just statistics….we’ve blobbed them all together and say that’s something’s a KPI when in fact, it’s a statistic. (Interview 2)|
|Tier 3: Emerging||There are different kinds of evidence. I could go out and find a bunch of unrelated information and say “This is my evidence”, but clearly that’s not it, is it? It has to be related to something in particular. (Interview 1)|
|Tier 4: Experimenting||We’re making business decisions based on a perfect world and data that has actually been cleansed intelligently, analysed appropriately and is tailored to actually answering the business question that is associated to a strategic goal or even an operational goal. (Interview 16)|
|Tier 5: Transforming||(Evidence) is more than data. It is information, so if you think about information that is within – contextualised, so within the context it’s valuable information that enables some decision making to take place. (Interview 14)|
Some library professionals in academic libraries have adopted evidence-based practice as an applied approach because they are driven by curiosity, aspire to do better, feel a professional responsibility, and want to keep up to date (Booth, 2002; Eldredge, 2000; Gillespie, 2014; Partridge et al., 2010). Understanding how a library as a whole fosters a culture of evidence-based practice provides insight into the various factors required to influence the ways in which evidence is conceived, encountered and used to improve service delivery and decision making. Regardless of how evidence-based practice is experienced as an individual phenomenon (Thorpe, Partridge, & Edwards, 2008), the variation of individual experiences is highly influenced by the workplace. Koufogiannakis’s (2015) study identified that organisational dynamics and the workplace context are the largest obstacle faced by academic librarians in applying evidence-based practice principles and processes. Library leadership and a customer-centred organisational culture were two of the enablers that supported the development of evidence-based practices in libraries (Hiller, Kyrillidou, & Self, 2008). A culture of evidence-based practice, requires all library staff, not just leaders, to appreciate the importance of the qualitative and quantitative evidence that they handle and to understand how such evidence can help demonstrate the value of library services (Urquhart, 2018). Shared organisational approaches to evidence-based practice, guide and develop library services and collections through an attitude and culture of continuous improvement (Gillespie et al., 2016).
As university libraries face increasing scrutiny of their role and value to the institution (Baker & Allden, 2017; Council of Australian University Librarians, 2016), responsibility for evidence-based practice approaches to service delivery and communication lies with the whole library organisation, not just individual practitioners. Individual librarians need organisational supports to enable evidence-based practice and related capabilities. Our research shows that the variation of experience, originally found by Partridge et al. (2010), appears to also be valid at an organisational level. The lived experience of USQ Library and the research interviews reinforce the role of the workplace context in building a culture of evidence-based practice (Howlett, 2018a). Library leadership in evidence-based practice is also essential to achieving organisational growth. The adoption of evidence-based practice should be achievable and aspirational at an organisational level, and not just reliant on the skills and interest of individual practitioners.
USQ Library is seeing the benefits of having explicit focus on building the capacity of library staff as evidence-based practitioners and developing tools to measure and sustain our success. These emerging initiatives demonstrate an applied approach by which research and academic libraries can become better informed and more adaptable to client and community needs in an ever-changing society. We have committed to establishing evidence-based practice as a focus and driver of excellence within our library service. The experience at USQ Library suggests that being evidence-based provides both intrinsic benefits to the library’s organisational culture and extrinsic benefits in demonstrating value and impact to stakeholders. Initial findings from research into evidence-based practice at the organisational level in academic libraries indicates that developing skills in the process, engagement and application of evidence may create a mature culture across all levels, not just among individual practitioners.