calculating valuethe nuts and bolts of implementing the university of colorado roi study
TRANSCRIPT
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
1/29
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
2/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Presentation Outline
Pilot Goals
Definitions
Methodology The Colorado Model
Data Management & Analysis
Recommendations
2
UCB Libraries
http://ucblibraries.colorado.edu/http://ucblibraries.colorado.edu/https://www.cu.edu/content/explore-cu-system -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
3/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Pilot Goals
Measure the impact of librarycollections on faculty researchproductivity
Find a metric to express libraryvalue to our administrators
Develop and test a costbenefit model for librarycollections and facultyresearch Office.com
http://office.microsoft.com/en-us/images/results.aspx?ex=2&qu=goals&AxInstalled=copy&Download=MP900387781&ext=JPG&c=0http://office.microsoft.com/en-us/images/results.aspx?ex=2&qu=goals&AxInstalled=copy&Download=MP900387781&ext=JPG&c=0 -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
4/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
ROI Studies
2009 Outsell and Springer study measured: Time saved by library users Money saved by using the library Revenue generated with the assistance of the library
IMLS Lib-Value Project Phase I: University of Illinois Urbana-Champaign
measured the value of the library in the grant process Phase II: International study measured value of e-
journals in academic libraries focusing on grantincome
Phase III: Lib Value: Values, Outcomes, and Returnon Investment of Academic Libraries
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
5/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Defining Value
Internal Focus
Use or Utility
Financial Value;
Return on Investment Commodity Production
External Focus
Impact Value
Alternative Comparison
Oakleaf, Value of Academic LibrariesOffice.com
http://office.microsoft.com/en-us/images/results.aspx?qu=fill&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=fill&ctt=1 -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
6/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
IRB Approval for Human Research
Librarians at eachcampus applied to theirInstitutional Review Board
(IRB) Requested and granted
Exempt Review
Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=approval&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=approval&ctt=1 -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
7/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Recruiting Participants
Auraria - focused on a single departmentrecommendation by Dean and support fromDepartment Chair
Health Sciences -focused on a singledepartment
Boulder - recruited
faculty from severaldepartments; usedfaculty library committee
Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=stairway&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=stairway&ctt=1 -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
8/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Methodology
Interviews
Curriculum vitaanalysis
Publication citationanalysis
Contingent Valuation
Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=meeting&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=meeting&ctt=1 -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
9/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Faculty Interviews
Conducted in-person,phone, or emails
Described pilot project
Requested copy ofcurriculum vita
Asked 4 questions regarding
library resource usageOffice.com
http://office.microsoft.com/en-us/images/results.aspx?qu=interview&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=interview&ctt=1 -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
10/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Interview Questions
1. In your most recently published article, forevery article you cited how many additionalarticles did you read but did not cite?
2. Approximately what percent of the articles citedor read were obtained from the following:
electronically from the library?
in print format from the library?
via Interlibrary loan from the library?
from non-library sources?
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
11/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Curriculum Vita Analysis andPublication Citation Analysis
Gathered publications (chapters & articles)published in 2009-2010
Identified cited references in faculty publications
Determined full-text sources for cited references
Calculated:
% cited references from journals
% of cited references from online libraryresources
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
12/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Contingent Valuation
Determine market valuefor articles
Random sample of journals
Cost to purchase individual
article from publisher Calculate average cost per
article
Calculate Standard Deviation
to determine low and highestimates
Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=stackhttp://office.microsoft.com/en-us/images/results.aspx?qu=stack -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
13/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Methodology
Interviews
Curriculum vitaanalysis
Publication citationanalysis
Contingent Valuation
Office.com
http://office.microsoft.com/en-us/images/results.aspx?qu=meeting&ctt=1http://office.microsoft.com/en-us/images/results.aspx?qu=meeting&ctt=1 -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
14/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
The Colorado Model
Based on Cost Benefit Analysis (CBA) andReturn on Investment (ROI)
CBAis the ratio showing dollar value of benefits
gained for dollar value of costs.Benefits Costs = CBA
ROI is the percentage showing the return orincrease in value on dollars spent to achieve a
benefit. ((BENEFITS COSTS) COSTS) x 100
14
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
15/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
University of Colorado Results
CU Boulder and CU Denver Auraria Library results
published in the proceedings for the 2011 Associationof College and Research Libraries Conference
available online at http://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdf
CU Denver Health Sciences Library results
forthcoming May 22 at the 2012 Medical LibrariesAssociation Annual Meeting
15
http://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.mlanet.org/am/am2012/http://www.mlanet.org/am/am2012/http://www.mlanet.org/am/am2012/http://www.mlanet.org/am/am2012/http://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdf -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
16/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Redefining Library Resources
Benefits = estimated cost for faculty to buy articlesdirectly from publisher with pay-per-view
Costs = Library costs for online journals/databases cited
CBA = Benefits Costs
ROI = ((BENEFITS
COSTS)COSTS) x 100
16
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
17/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Colorado Model with Hypothetical Data
Benefits = estimated cost for faculty to buy articlesdirectly from publisher with pay-per-view
Estimated value of articles: $15,441
Costs = Library costs for online journals/databases cited
Journal subscriptions: $15,164
CBA = Benefits Costs
$15,441 $15,164 = $1.02
ROI = ((BENEFITS
COSTS)COSTS) x 100
(($15,441-$15,164) $15,164) x 100 = 2%
17
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
18/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Data Management & Analysis
University of Colorado Model
Sample implementation with hypothetical data
Handout summarizes CBA and ROI formulas, variables and values
Excel workbook demonstrates
citation analysis and implementing model
18
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
19/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Handout
Defines formulas
Presents variables
Identifies data sources
Faculty CVs
Citation Analysis
Interview Answers
Cost Estimates
#buildROI 19
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
20/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Excel worksheet: Colorado Model
Highlighted totalvalues for variableslink to corresponding
worksheet E.g. Variable A
(yellow) = 15 articlespublished in 1 year
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
21/29
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
22/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Excel worksheet: Citations
22
Data elements for citation analysis Participants cited references perpublished articles
Cited references sources (Journal X)
Full-text source (Database Y)
Acquisition type (Online Subscriptionv. Open Access or No Access)
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
23/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Excel worksheet: I
23
Estimated Cost per ArticleBased on sampling of randomlyselected titles Utilizes pivot table of citation
analysis data Includes formulas forrandomly selecting titles
Look up pay-per-view priceson publishers website
Calculates costs for average
and low/high standarddeviation
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
24/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Excel worksheet: K
24
Estimated Library Costs forSubscriptions based on cost tosubscribe to referenced titles
Utilizes pivot table to identifyfull-text source
Look up subscription costs andtotal # of titles in full-text sourceto derive avg. price per title
Calculates library costs bymultiplying avg. price per titlewith # of journals referenced infull-text source.
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
25/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Analysis of Article Price Estimates
25
Contingent valuationis market value ofarticle costs
More statistically
significant by Random sample
of titles
Standard deviation ofaverage article costs
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
26/29
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
27/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Implementation Recommendations
Align project to mission and objectives ofthe library and institution
Demonstrate support of faculty research
Consider readily available data sources
Access to CVs, course syllabi, faculty report ofprofessional activities, grant funding
Define scope and focus of project
Specific department and number of participants
Identify manageable timeframe Multiple steps (IRB approval, interviews, citation analysis)
27
-
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
28/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Resources
Association of College and Research Libraries. (2010). Value of Academic Libraries: AComprehensive Research Review and Report. Researched by Megan Oakleaf.Chicago: Association of College and Research Libraries. Available online at:www.acrl.ala.org/value
Barnes, S. & Blake, M. (2010). Measuring Your Impact: Using Evaluation for LibraryAdvocacy, A library evaluation workshop. National Network of Libraries of Medicine,http://nnlm.gov/evaluation/workshops/measuring_your_impact/.
Kaufman, P. (2008). The library as strategic investment: Results of the Illinois return on
investment study.Liber Quarterly, 18(3/4), 425. Luther, J. (2008). University investment in the library: Whats the return?: A case study at
the university of Illinois at UrbanaChampaign. Available online at:http://libraryconnect.elsevier.com/whitepapers/0108/lcwp0101.pdf.
Pan, D., Wiersma, G., & Fong, Y. (2011). Towards demonstrating value: Measuring thecontributions of library collections to university research and teaching goals.Proceedings of the 2011 ACRL Conference Proceedings. Available online at:http://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/
papers/towards_demonstratin.pdf Tenopir, C. (2012). Beyond usage: measuring library outcomes and value. Library
Management, 33(1/2), 5-13.
28
http://www.acrl.ala.org/valuehttp://nnlm.gov/evaluation/workshops/measuring_your_impact/http://libraryconnect.elsevier.com/whitepapers/0108/lcwp0101.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://www.ala.org/acrl/files/conferences/confsandpreconfs/national/2011/papers/towards_demonstratin.pdfhttp://libraryconnect.elsevier.com/whitepapers/0108/lcwp0101.pdfhttp://libraryconnect.elsevier.com/whitepapers/0108/lcwp0101.pdfhttp://nnlm.gov/evaluation/workshops/measuring_your_impact/http://nnlm.gov/evaluation/workshops/measuring_your_impact/http://www.acrl.ala.org/value -
8/12/2019 Calculating ValueThe Nuts and Bolts of Implementing the University of Colorado ROI Study
29/29
University of Colorado
Boulder | Denver | Anschutz Medical Campus
Questions? Thank You!
Gabrielle [email protected]
Leslie [email protected]
Denise [email protected]
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]