measuring value and outcomes of reading dr. carol tenopir university of tennessee [email protected]
TRANSCRIPT
![Page 2: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/2.jpg)
What information they need and use
How many articles they read
Changes in behavior, formats, needs
Purpose and outcome of readings
Value of libraries and publishers
Ongoing studies of scientists…(Audiences=librarians, funders, publishers, scientists)
![Page 3: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/3.jpg)
Methods for measuring value
Usage Citations
Focus groups
Return on Investment
Perceptions
Experiments Critical incident Contingent valuation
Observations
Conjoint measurement
Interviews
![Page 4: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/4.jpg)
Critical Incident Technique
• Surveys by Tenopir and King (1977-present)• Ask respondents about most recent reading• Ask questions on purpose, motivation and
outcomes of specific reading• In-depth picture of complexity of readings
![Page 5: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/5.jpg)
“The following questions in this section refer to the SCHOLARLY ARTICLE YOU READ MOST RECENTLY, even if you had read the article previously. Note that this last reading may not be typical, but will help us establish the range of patterns in reading.”
Critical Incident
![Page 6: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/6.jpg)
Definitions
“Articles can include those found in [e] journal issues, web sites, or separate copies such as preprints, reprints, and other electronic {or paper} copies.
Reading is defined as going beyond the table of contents, title, and abstract to the body of the article.”
Carol Tenopir
![Page 7: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/7.jpg)
Principal Purpose of Reading (Faculty in U.S. and Australia, 2004-2006, n=1433)
51%
9%
11%
9%
20%
Research
Teaching
CurrentAwareness
Proposals
Other
![Page 8: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/8.jpg)
Source of reading by purpose of reading by faculty
(Faculty in U.S. and Australia, 2004-2005, n=1412)
ResearchTeaching
Current Awareness
5%
5% 5%
11%
37%
38%
3%3%
30%
47%
7%
10%
3%
50%
18%
14%
7%
9%Library provided
Personalsubscription
Open Web
Colleagues
SchooldepartmentsubscriptionOther
![Page 9: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/9.jpg)
Library E-Collections are Most Common Source of Additional Readings
37
113
52
120
92 96 101115
131121
0
20
40
60
80
100
120
140
1977 1984 1993 2000-03 2004-06
Year of Survey
Scientist Readings Per Year
Library collection Other
![Page 10: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/10.jpg)
Readings for Research
• More likely to be rated “absolutely essential”
• More likely to be found by searching
• More likely to be from e-sources
• More likely to be from the library
![Page 11: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/11.jpg)
Comments (2008) tell us that E-Collections improve…
• Efficiency and productivity“[e-access] saves me a lot of time which can be
used for more extensive reading.”
• Writing and proposals“[E-access] is essential for scientific writing.”
• Research and teaching“I could not do the kind of research or teaching I do
without these resources.”
![Page 12: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/12.jpg)
Average number of articles read annually by publishing productivity
(number of articles published in the past 2 years) (Faculty in U.S. and Australia, 2004-2005, n=1364)
0
50
100
150
200
250
300
350
400
Leastproductive
(0-1)
Mediumproductive
(2-5)
Mostproductive
(over 5)
![Page 13: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/13.jpg)
Principal purpose of reading by publishing productivity
(number of articles published in the past 2 years)
(Faculty in U.S. and Australia, 2004-2005, n=1366)
0%
10%
20%
30%
40%
50%
60%
70%
Research Teaching Currentawareness
Writing Other
Leastproductive (0-1)
Mediumproductive (2-5)
Mostproductive(over 5)
![Page 14: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/14.jpg)
How important is the information contained in
this article to achieving your principal purpose?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
Research Teaching Administration CurrentAwareness
WritingProposals
Not at allimportant
Somewhatimportant
Absolutelyessential
![Page 15: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/15.jpg)
How important is the information contained in
this article to achieving your principal purpose?
0%
10%
20%
30%
40%
50%
60%
70%
Library Source Personal Source Others
Not at allimportant
Somewhatimportant
Absolutelyessential
![Page 16: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/16.jpg)
National differences
Carol Tenopir
Faculty in Australia have the highest amounts of e-reading and report a significant amount (6.7%) of readings done while travelling
Faculty in Finland access articles more at the office or lab, less at home
Finnish scholars report a higher use of searching in locating e-articles
Faculty in Taiwan, Japan and Finland have a higher percentage of readings from the library
![Page 17: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/17.jpg)
Return on Investment (ROI)ROI is a quantitative measure expressed as a ratio
of the value returned to the institution for each monetary unit invested in the library.
For every $/€/¥ spent on the library,the university received ‘X’ $/€/¥ in return.
Demonstrate that library collections contribute to income-generating activities
![Page 18: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/18.jpg)
Study in 3 Phases
Phase 1: ROI in grants, case study at a U.S. University (completed 2008)
Phase 2: ROI in grants, expanded to 8 countries (completed 2010)
Phase 3: ROI for grants/research, teaching, student engagement (in progress now)
![Page 19: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/19.jpg)
Types of Data Collected
Data sources Method
Research Faculty Survey; quantitative and qualitative
Grant Proposals Univ research data; survey
Grant Income Univ research office data
Library Total budget (including collection, facilities, personnel, etc.)
Administrators’ priorities Interviews (with library, university, and research administrators)
![Page 20: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/20.jpg)
Phase 28 institutions in 8 countries
![Page 21: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/21.jpg)
Phase 2: ROI for Grants
![Page 22: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/22.jpg)
Phase 2: Why Does ROI for Grants Vary?
• ROI depends on institutional mission• Research institutes have very large grants• Teaching universities have smaller and fewer
grants• ROI varies depending on methods of
government funding• Be cautious comparing ROI across institutions
![Page 23: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/23.jpg)
Ranking and Conjoint Analysis
When faced with numerous articles on their topic, how do scholars make the decision of which articles to read?
A study funded by:
![Page 24: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/24.jpg)
Methodology
•International online survey of scholars
•Ranking and conjoint analysis provide relative importance of various article characteristics
•Additional questions to help identify differences: amount of reading, subject discipline, age, and location
![Page 25: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/25.jpg)
The survey yielded 442 responses from 12 countries
![Page 26: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/26.jpg)
Ranking Article Characteristics
Respondents were asked to rank the importance of seven article characteristics from 1-7 where 1 is “most important” and 7 “least important.”
26
![Page 27: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/27.jpg)
0
100
200
300
400
1 2 3 4 5 6 7Rank
Top Choice: Article Topic
27
![Page 28: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/28.jpg)
Next most important were:
Online accessibility (3.27)
Source of article (3.46)
Journal Title (3.58)
Author(s) (4.07)
28
![Page 29: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/29.jpg)
020406080
100120
1 2 3 4 5 6 7
Rank
Online Accessibility (Score 3.27)
29
![Page 30: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/30.jpg)
0
20
40
60
80
100
120
1 2 3 4 5 6 7
Rank
Source of Article (Score 3.46)
30
![Page 31: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/31.jpg)
0
20
40
60
80
100
120
1 2 3 4 5 6 7
Rank
Journal Title: (Score 3.58)
31
![Page 32: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/32.jpg)
0
20
40
60
80
100
1 2 3 4 5 6 7
Rank
Author(s): (Score 4.07)
32
![Page 33: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/33.jpg)
The least important:
Type of Publisher (5.21)
Author’s Institution (5.82)
33
![Page 34: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/34.jpg)
020406080
100120140
1 2 3 4 5 6 7
Rank
Type of Publisher: (Score 5.21)
34
![Page 35: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/35.jpg)
0
50
100
150
200
1 2 3 4 5 6 7
Rank
Author(s)' Institution: (Score 5.82)
35
![Page 36: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/36.jpg)
Academic Disciplines16%
19%
17%
28%
9%
11%
HumanitiesSciencesMedical/HealthSocial SciencesEngineeringProfessional
n=41236
![Page 37: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/37.jpg)
Humanities Sciences Medical Social Sciences
Engineering Professional
Topic 1 1 1 1 1 1
Online Accessibility
3 2 2 2 2 4
Authors 5 5 5 5 5 5
Institution 7 7 7 7 7 7
Source (journal)
2 4 3 3 4 2
Type of Publisher
6 6 6 6 6 6
Journal Title 4 3 4 4 3 3
Ranking of Article Characteristics by Discipline
37
![Page 38: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/38.jpg)
Conjoint Analysis Methodology
•Respondents considered16 article profiles with each consisting of a different permutation of article characteristics.
•They rated how likely they were to read an article with that profile on a scale of 1-10 ( 1 is “absolutely would not read” and 10 is “absolutely would read.”)
•For conjoint analysis, the 16 profiles explored three characteristics: Author, Journal Type, and Accessibility.
![Page 39: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/39.jpg)
Profile 1 How likely are you (on a scale of 1 to 10) to read an article with these characteristics?
Written by an author I recognize as a good (but not top) scholar, in a peer-reviewed journal outside the top tier, and available online at no personal
cost [to reader].
Absolutely would not read-→1…2…3…4…5…6…7…8…9…10←-Absolutely would read
Profile 13How likely are you (on a scale of 1 to 10) to read an article with these characteristics?
Written by an author I recognize as a good (but not top) scholar, from a source other than a journal, and available online at no personal cost.
Absolutely would not read-→ 1…2…3…4…5…6…7…8…9…10 ←-Absolutely would read
39
![Page 40: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/40.jpg)
Author Utility Estimate
Top Tier 1.412 Good .386 Weak -1.193 Unknown -.605
40
![Page 41: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/41.jpg)
Journal Utility Estimate
Top Peer 1.113 Other Peer .376 Not Peer -.971 Not Journal -.519
41
![Page 42: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/42.jpg)
Access Utility Estimate
Online no personal Cost 1.604 Online personal Cost -1.107 Print Only -.497
42
![Page 43: Measuring Value and Outcomes of Reading Dr. Carol Tenopir University of Tennessee ctenopir@utk.edu](https://reader033.vdocument.in/reader033/viewer/2022042822/56649eb55503460f94bbe270/html5/thumbnails/43.jpg)
Some Final Thoughts on Measuring Value
• Measure motivations, purposes, outcomes
• Variations by purpose and discipline
• No one method stands alone