Value and Impact of University Libraries Thursday, 25 September 2014

The following survey of resources has been compiled by CAUL's Quality & Assessment Advisory Committee (CQAAC):

The context for many of the resources listed here arises from the need for libraries to prove their value:
Not only do stakeholders count on higher education institutions to achieve their institutional goals, they also require them to demonstrate evidence that they have achieved them. The same is true for academic libraries; they too must provide evidence of their value (Oakleaf  2010, p. 26).

INTRODUCTION

For about 10 years there has been significant debate across the library profession on the merit of retaining traditional measures and the growing need for a new, alternative methodology. For the sake of simplicity the traditional method may be characterized as ‘counting’ – the size of collections, the size of the budget, the size of staffing complement, the number of issues/ downloads, and various ratios of collections & expenditure. Underpinning this is an assumption that bigger or more equates to better. However, in recent years newer measures have been promoted as qualitatively worthier illustrations of the use of the library by the communities served. This has shifted focus from ‘size’ to ‘impact’, and as newer suggests this necessitates the collection of other data. The emphasis has, therefore, changed from statistical data on user transactions to information about user needs and library services aligned to meet those needs. The scope of ‘new’ information includes: data from rubrics to assess the effectiveness of information literacy on outcomes; data from direct interaction with faculty & students for instance via Liaison Librarians; composite data for profiling of market segments (including personas & users journeys). The purpose here is not only to illustrate that (some) library resources have been used (quantitative data) but that resources & services are relevant and may be correlated to outcomes (qualitative information). This supports a shift of focus to the tailoring or customisation of services to ensure that outputs are aligned with the needs of various user groups.

BACKGROUND

A bibliography prepared by Margie Jantti & Karen Tang for CAUL (CQAAC) in 2011, surveyed resources on Return On Investment And Value Of Libraries.The following is a compendium of significant projects & papers that have focussed upon ‘new measures’, ‘impact of library services’, and ‘value’ and has been compiled in keeping with the scope of, and incorporating, the original bibliography prepared by Margie Jantti (Wollongong) & Karen Tang (Curtin). Additional material that a) reflects how libraries can demonstrate their value & impact, and b) the extent to which libraries should position themselves to be valued in their communities has been provided by Eamon Wright (Auckland) .

CONFERENCES

Measuring & Improving Library Value: defining indicators of quality & impact
A two-day conference, held in Sydney in late 2012, which focused upon the need for libraries to demonstrate their organisational value to host institutions particularly in the context of current, largely, economic pressures. Although the majority of attendees were from the Australian university library sector the theme of this conference resonated with other library types including State, City and special libraries who were also represented. A recurrent proposition was that stakeholder needs is vital business intelligence that should inform provision of library services including marketing/ communication/ PR strategies. This suggests that methodologies for understanding those needs must be developed: modelling and delivering services on users’ needs requires input from users in the first instance.
 
The Northumbria International Conference on Performance Measurement in Libraries and Information Services
The Northumbria International Conference in the United Kingdom has taken place biennially since the early 1990s. This is a premier platform of library & information professionals. Established experts or new and enterprising proponents of developments in measurement & impact, showcase recent projects, reports and work-in-progress. 

PROJECTS

Library Analytics and Metrics Project (LAMP)
This is a United Kingdom, JISC-funded project that aims to “enable libraries to capitalise on the many types of data they capture in day-to-day activities, using this to support the improvement and development of new services and demonstrate value and impact in new ways across the institution”.
A new, in 2014, LAMP project (paper forthcoming) includes comments linking Huddersfield/LAMP with Wollongong – see blog: http://jisclamp.mimas.ac.uk/
Library Impact Data Project (LIDP)
This was a project led by Graham Stone, Dave Pattern, and Bryony Ramsden, University of Huddersfield in the United Kingdom, that aimed “to prove a statistically significant correlation between library usage and student attainment“. It has been superseded by LAMP.
 
Information on the aims and findings from each phase and ‘going forward’ considerations can be found:
Oakleaf , Megan et al 2013, ‘Do or Do Not...There is No Try: The Quest for Library Value’. In: Association of College & Research Libraries Conference, 10-13 April 2013, Indianapolis, Indiana.
 
Detail of phase I of the project, its background and its findings can be found from the following publications:
Stone, Graham & Ramsden, Bryony 2013, 'Library Impact Data Project: looking for the link between library usage and student attainment', College and Research Libraries,vol. 74, no.6 pp. 546-559.
 
Final report covering phase I and II of the project can be found:
Kay, David & Harmelen, Mark van 2013, Activity data: delivering benefits from the data deluge, JISC, UK.
LibValue
This is an ILMS funded grant project to enable libraries to demonstrate and uncover the many way in which they contribute to institutional success.
Rubric Assessment of Information Literacy Skills (RAILS)
This United States initiative, led by Megan Oakleaf, has developed a series of rubrics or guides to assess information literacy outcomes. The tools are tabular templates that include specific attributes of information literacy competencies some of which suggest applicable marking-scales.

PUBLICATIONS

Aabø, Svanhild 2005, ‘The role and value of public libraries in the age of digital technologies’, Journal of Librarianship and Information Science, vol. 37, no. 4, pp. 205-211.
Discusses public libraries’ role and value in the age of digital technologies. Reassessments of their role due to technological development and widespread public use of the Internet are analysed. Central challenges of the digital society, including an increased digital divide and a weakening of local community identity, have resulted in lower social participation and involvement in community issues. Previous research has demonstrated that public libraries have a wide social impact on both individuals and local communities. This article focuses on the special characteristics of public libraries to assess their potentially enhanced role and value, as a public room and social and physical meeting place in the digitised age. The article identifies a need to strengthen the public libraries’ democratic role in the information society by furthering social inclusiveness and citizenship.
 
Aabø, Svanhild 2009, ‘Libraries and return on investment (ROI): a meta-analysis’, New Library World, vol. 110, no. 7/8, pp. 311-324.
The purpose of this paper is to show that the need to communicate the value of libraries is growing, and especially now during the global financial crisis. As a response library valuation research is expanding and there is now a need for a status report. The library valuation field is developing towards generating a critical mass of empirical studies. The focus of the meta-analytical review is on the subgroup that reports a return on investment (ROI) or a cost-benefit ratio. Meta-analysis is a quantitative analysis of findings of previous studies, conducted to infer general findings and lessons from prior empirical research. The dataset is 38 library valuation studies reporting a return on investment figure or cost-benefit ratio. Of the 38 studies, 32 are of public libraries, a number high enough to indicate a tenable result. The meta-analysis indicates that the patterns in the findings are consistent with expectations regarding the benefit types that are included in the ROI figure, the methods used, and the scope of the study. This study appears to be the first meta-analytical review of library studies reporting a return on investment figure. The tentative conclusion is that for each dollar invested in public libraries they return, on average, approximately four times more. This is a strong message with policy implications.
 
Brown, Karen & Malenfant, Kara J 2012, Connect, Collaborate, and Communicate: A Report from the Value of Academic Libraries Summits, Association of College and Research Libraries, Chicago.
This report, which is very much linked to Value of Academic Libraries: A Comprehensive Research Review and Report, summarizes the ACRL landscape. To this end, the authors suggest that “academic librarians can serve as connectors and integrators, promoting a unified approach to assessment. As a neutral and well-regarded place on campus, the academic library can help break down traditional institutional silos and foster increased communication across the institutional community. Librarians can bring together people from a wide variety of constituencies for focused conversations and spark communities of action that advance institutional mission.” (p. 16)
 
Chung, Hye-Kyung 2007, ‘Measuring the economic value of special libraries’, The Bottom Line: Managing Library Finances, vol. 20, no. 1, pp. 30-44.
This study aims to present a new approach to measuring the economic value of special libraries, including certain time-saving effects that the contingent valuation method application cannot exclusively prove. A cost-benefit analysis is used as a tool to determine whether the benefits of special libraries outweigh the cost incurred in providing the services. The benefits of such libraries are based on estimates of how much the user is willing to pay for the service, as well as the cost of time saved as a result of his contact with library services. A case study was conducted to show how special libraries could apply the proposed model to their library setting to measure the value of the library's services. According to the case study involving the KDI School Library, the economic value of its library services measured in terms of a B/C ratio was 1.97, serving as strong justification for the library's existence. This study is more specific and accurate than previous studies in that it enables an individual analysis for each service special libraries offer and focuses on the types of benefit derived. It is hoped that the model will help analyze the strength of each library service as well as the total economic value of the library.
 
Elliott, Vic 2010, ‘”Why Then We Rack the Value”- Building Value Frameworks for Academic Libraries’ Proceedings of the ALSR 2010 Conference towards Future Possibilities, Academic Librarian 2: Singing in the Rain,
 
Grzeschik, Kathrin 2010, ‘Return on investment (ROI) in German libraries: The Berlin School of Library and Information Science and the University Library at the Humboldt University, Berlin – a case study’, The Bottom Line: Managing Library Finances, vol. 23, no. 4, pp. 141-201.
The purpose of this paper is to verify the proposition by the University of Illinois, Urbana-Champaign (UIUC), that their return on investment (ROI) formula developed for academic libraries and based on hard facts is broad enough to be used throughout the world for ROI studies in academic institutions/libraries. It further aims to verify that UIUC's methodology is adaptable enough to work in other academic environments as well.
 
Hendriks, Boyd & Wooler, Ian 2006, ‘Establishing the return on investment for information and knowledge services: A practical approach to show added value for information and knowledge centres, corporate libraries and documentation centres’, Business Information Review, vol. 23, no. 1, pp. 13-25.
 
Jantti, Margie & Cox, Brian 2010, ‘Measuring the value of library resources and student academic performance through relational datasets’ Proceedings of the Library Assessment Conference: Building Effective, Sustainable, Practical Assessment, 25-27 October, Baltimore, Maryland.
 
Jemison, Karen et al 2009, ‘Measuring Return on Investment in VA Libraries’, Journal of Hospital Librarianship, vol. 9, no. 4, pp. 379-390.
After evaluating available library-targeted return on investment (ROI) tools, VA Library Network Librarians created a workgroup to formulate a tool geared specifically to the Department of Veterans Affairs (VA) hospital libraries. The workgroup devised an approach with three separate components: a Return on Investment Analysis tool that each library could employ; a Library Scorecard that provided local and national examples of typical benefits, for libraries that could not devote time to using the ROI instrument; and a Management Support Report for libraries that wanted a mission-based document to use with their local management. The tools were essentially complete in 18 months, after numerous drafts.
Kaufman, Paula & Watstein, Sarah Barbara 2008 ‘Library value (return on investment, ROI) and the challenge of placing a value on public services’, Reference Services Review, vol. 36, iss: 3, pp.226 – 231.
During 2007, a small project team was assembled to develop a model that would calculate a return on investment to an institution for its library. The results of this ground-breaking study were reported in a white paper entitled, "University investment in the library: what's the return? A case-study at the University of Illinois at Urbana-Champaign" by Judy Luther. The model the team developed showed a 4-to-1 return. http://dx.doi.org/10.1108/00907320810895314
 
Luther, Judy 2008, ‘University investment in the library: What’s the return? A case study at the University of Illinois at Urbana-Champaign’, LibraryConnect White Paper #1, Elsevier.
 
Missingham, Roxanne 2005, ‘Libraries and economic value: a review of recent studies’, Performance Measurement and Metrics, vol. 6, no. 3, pp. 142-158.
This paper aims to outline the development of research into the value of libraries over the past decade. Recent studies using contingent valuation for the British Library, South Carolina Public Libraries, Florida Public Libraries and St Louis Public Libraries are summarised both in terms of methodology and findings. Studies into two national bibliographic services (Canada and New Zealand) are reviewed to demonstrate the application of value studies to specific services. There are many questions that have yet to be answered through using this methodology. At the most basic level it is not yet clear whether any particular numerical result represents the best return on investment for an individual library. The lack of comparative of studies means that the appropriate level of return on investment than that which the taxpayer or investor should expect, has yet to be established. There is a need for further research to identify the relative position in which libraries in the major sectors should expect to be found. More significantly, there is a need to consider how a value identified for current use of a service should be balanced against future use, and to establish how these two analyses might be combined. Shows consistent use of contingent valuation and return on investment for libraries in public and national library sectors. Each study took considerable resources and man-hours to establish a community/user based economic result.
 
Neal, James G. 2011, ‘Stop the madness: the insanity of ROI and the need for new qualitative measures of academic library success’, Proceedings of the ACRL 2011 Conference, A declaration of Interdependence, March 30 - April 2, 2011, Philadelphia.
 
Oakleaf, Megan 2010, Value of Academic Libraries: A Comprehensive Research Review and Report, Association of College and Research Libraries, Chicago.
This report describes the current state of the research on university library value within an institutional context and suggests focus areas for future research around articulating academic library value. Although slightly dated now, it still provides a good summary of what kind of evidence would help tell a library’s story and how best to gather that evidence.
 
OCLC 2010, Perceptions of Libraries, 2010: Context and Community: a Report to the OCLC Membership, OCLC, Dublin, Ohio.
OCLC has revisited an earlier 2005 study on how libraries are perceived, to analyse the impact of changed information technologies and sources as well as the recession. Identifying library use and expectations, and how these vary by generation (college students, young adults, and generation X) is the report’s particular strength, showing how services can be best targeted and justified. In general its findings indicate that the value of libraries and librarians, not just in terms of individual use but also for a community, has been increased by the recession and as unemployment levels rise. The role of libraries in supporting retraining, continuing education, and new careers is evident, with the 46-64 age group for example, ‘using the library more often for job-related activities, training or educational programs, and seeking college-related information’. In concluding with possible trends the report argues that there is continuity too. Even though forms of reading may change ‘libraries are even more about books’, with the building of collections and extending opening hours being among the survey’s top requests.
 
Paberza, Kristine 2010, ‘Towards an assessment of public library value: Statistics on the policy makers' agenda’, Performance Measurement and Metrics, vol. 11, no. 1, pp. 83-92.
This paper aims to present a methodology, early findings, possible applications of results and lessons learned from the research study "Public libraries: value, trust and satisfaction", which has been conducted within the public library development project 'Father's Third Son' in Latvia.
 
 

Poll, Roswitha 2014, Bibliography ‘Impact and Outcome of Libraries’, ULB WWU,Munster, Germany.
This is an extensive list of projects, publications, and website resources on a range of value facets and contexts.

http://www.ifla.org/files/assets/statistics-and-evaluation/publications/bibliography_impact_and_outcome_2014.pdf

 

 
 
Poll, Roswitha  2012, ‘Can we quantify the library’s influence? Creating an ISO standard for impact assessment’, Performance Measurement and Metrics, vol. 13, iss. 2, pp. 121-130.
 
Research & Information Network 2011, The Value of Libraries for Research and Researchers: a RIN and RLUK report, RIN/RLUK, London, UK.
The report of a UK-based project which reviewed the contribution which academic libraries make to HE whether generally in helping to maintain a research profile, raise research income, and exploit new technology, or specifically in supporting individual researchers and the environment in which they can be most productive. There is a familiar dilemma that libraries should explain how they are changing but also that they should not be judged solely by the immediate needs they serve. In concentrating on the value of academic libraries specifically for research it associates libraries with institutional strength and acknowledges the importance of specialist library staff with subject and research expertise. Above all it recognises the inclusive academic values which libraries represent: ‘the scholarly ethos that universities exist to inculcate and preserve’.
 
Sidorko, Peter Edward 2010, ‘Demonstrating ROI in the library: the Holy Grail search continues’, Library Management, vol. 31, no. 8/9, pp. 645-653.
This article aims to examine approaches by academic libraries in demonstrating return on investment (RoI). As a participant in a recent international RoI study, the author reviews the various difficulties in developing a suitable methodology. Using grant income as the basis for demonstrating RoI, it was found that wide differences in results may be attributable to a number of factors related to the parent organisation, the availability of grant funding and the country of the study. Further work is necessary to arrive at a suitable methodology for a diverse range of academic libraries. Library managers are alerted to issues and problems surrounding the development of return on investment methodologies. This paper will prove useful to librarians considering investing time and other resources in developing methodologies for demonstrating return on investment.
 
Sumsion, John; Hawkins, Margaret & Morris, Anne 2002, ‘The economic value of book borrowing from public libraries: An optimisation model’, Journal of Documentation, vol. 58, no. 6, pp. 662-682.
In the context of statistical research into the economic value of public library services, a model was developed to demonstrate the economic benefit when books are borrowed rather than bought. The model is based on the number of book reads rather than on book purchases or library issue counts. Different assumptions applied to the model cover the hardback/paperback distinction and different levels of library costs. The most significant variable, however, is shown to lie between books that are "read through" and those `frequently consulted" for information and educational benefit. Maximising book loans through the public library is shown to be not only in the interest of individual users, but also to be economically in the public interest.
 
Sumsion, John; Hawkins, Margaret & Morris, Anne 2003, ‘Estimating the economic value of library benefits’, Performance Measurement and Metrics, vol. 4, iss: 1, pp.13-27
The theory underlying the economic value of library benefits is outlined, and research (mainly in Australia and New Zealand) is reviewed. A UK research project examined four methods of assessing benefits in economic terms with particular attention to a consensus "market value" model. In developing the "market value" model one key variable is the relationship of book reads to book prices. A prototype value added schedule gives estimates of value for different library services to compare estimated total benefits with total costs. For UK public libraries, calculations show that the economic value of library benefits exceeds costs incurred, with social and intangible benefits in addition. New performance indicators are suggested by the research. It is shown how the methodology can be extended from public libraries to a parliamentary library and also to the economic and social costs of crime.
 
Tenopir, Carol 2010, ‘Measuring the Value of the Academic Library: Return on Investment and Other Value Measures’, The Serials Librarian, vol. 58, iss. 1-4.
 
Tenopir, Carol et al2011, ‘Perceived Value of Scholarly Articles’, Learned Publishing, vol. 24, no. 2, pp. 123-132.
When faced with an abundance of articles, readers must weigh the relative importance of various characteristics to select which articles to read. Over 400 researchers in 12 countries responded to a questionnaire that asked them to rank seven article characteristics and rate 16 article profiles. After article topic, the next most highly ranked characteristics were online accessibility and source of article. Conjoint analysis revealed the highest rated profiles to be (i) article written by a top-tier author, in a top peer-reviewed journal, available online at no personal cost to the reader; and (ii) article written by a top-tier author, in a peer-reviewed journal not in the top tier, available online at no personal cost to the reader. There were significant differences in characteristic rankings by discipline and geographic location.
 
Weiner, Sharon A 2005, ‘Library Quality and Impact: Is There a Relationship between New Measures and Traditional Measures?’ The Journal of Academic Librarianship, vol. 31, no. 5, pp. 432-437.
This paper by provides historical context to the discussions across the profession about the relationship between library services and factors related to expenditures, types of clientele, and staffing.

WEBSITES

Association of College and Research Libraries

Association of College and Research Libraries (ACRL) is a division of the American Library Association. As part of its advocacy strategy, the ACRL established the Value of Academic Libraries initiative, to ensure that academic libraries demonstrate alignment with and impact upon institutional outcomes. It includes information on products like MINES and LibValue.

http://www.arl.org/focus-areas/statistics-assessment 

The ACRL conference is an opportunity for sharing practices, skills and tools”

http://libraryassessment.org/