niso webinar on usage data: an overview of recent usage data research

NISO Webinar on Usage Data

An Overview of Recent Usage Data Research

John McDonaldLibraries, Claremont University

ConsortiumMay 13, 2009

Increased Interest in Usage Data• Ability to measure actions

• Usage

• Citation

• Relationships between resources

• Ability to analyze large datasets

• Computational power

• Data provided directly to librarians

• Standards for data and distribution

• Ability to demonstrate return on investment

• Management data

• Collections data

New Ways to Collect Usage Data

• ISI Citation Data

• COUNTER reports

• Publisher provided data

• Web server logs

• Proxy server logs

• OpenURL resolver logs

• Google Analytics

Theoretical Analysis of Usage Data

• Bollen’s Centrality Measures

• Rosvall & Bergstrom’s Scientific Communication Maps

• Davis’ Open Access studies

Citation and Usage Data MeasuresBollen, Van de Sompel, Hagberg, Chute (2009). A principal component analysis of 39 scientific impact measures. arXiv. Available: http://arxiv.org/PS_cache/arxiv/pdf/0902/0902.2183v1.pdf

• A study of 39 journal measures, both standard bibliographic measures derived from citation and other measures derived from usage.

• Outcomes included that citation and usage are distinctly different events and measures based on them do not correlate closely.

Figure 2 from Bollen et. Al. (2009).Usage based

measures

Citation Based

Measures

Illustration of Citation NetworksRosvall & Bergstrom (2008). Maps of random walks on complex networks reveal community structure. PNAS Available: http://octavia.zoology.washington.edu/publications/RosvallAndBergstrom08.pdf

• A scientific map of the citation relationships between 6000+ ISI-indexed journals.

• Outcomes indicate that many basic science fields have bidirectional relationships with other fields, while most applied fields have uni-directional relationships with the basic science fields.

Figure 3 from Rosvall & Bergstrom

• bergstromReciprocal

citation relationship

Non-reciprocal citation

relationship

Analysis of Open Access citationsDavis (2008). Author-choice open access publishing in the biological and medical literature: a citation analysis. arXiv. Available: http://arxiv.org/PS_cache/arxiv/pdf/0808/0808.2428v3.pdf

• A study of 11journals where open access status was assigned randomly to articles to determine the citation advantage for OA articles.

• Outcomes included that OA articles were not more likely to accumulate citations than paid access articles.

Table S2 from Davis (2008)

Evidence Based Analysis of Usage Data• Betty’s Google Analytics of Local

Content

• Grigson’s Analysis of eBook Models

• Kinman’s Use of Sparklines

Analyzing Local Usage Data• Betty (2009). Assessing Homegrown Library Collections:

Using Google Analytics to Track Use of Screencasts and Flash-Based Learning Objects. Journal of Electronic Resources Librarianship. Volume 21:1, 75 – 92.

• A study utilizing Google Analytics to track the use of web-based tutorials for library instruction.

• Outcomes included information about the total hits to each tutorial, usage throughout a tutorial, connection speed, browser software components

Betty’s Tutorials Usage Results• One tutorial had 23% of its hits recorded by an

unintended audience

• Possible action: Better marketing/description of the content

• High hits for beginning & end of tutorials

• Possible action: Shorten or revise content in areas being skipped

• Most users had necessary software to view files

• Possible action: None needed

• A significant minority of users had dial-up access

• Possible action: Produce multiple versions

Evaluating eBook Usage Data• Grigson (2009). Evaluating Business Models for E-Books

Through Usage Data Analysis: A Case Study from the University of Westminster. Journal of Electronic Resources Librarianship. Volume 21:1, 62-74.

• A study comparing usage of ebook packages provided by vendors with different acquisitions models (simultaneous users v. annual usage)

• Outcomes resulted in a clearly preferred model focusing on annual usage to accommodate the high peaks of usage during academic semesters.

Table 1 from Grigson’s eBookUsage

Indicates peak-use periods of high demand for portions of the collections

Evaluating Ebook Usage Data

Kinman (2008). Putting the Trees Back in the Forest: E-Resource Usage Statistics and Library Assessment. ER&L, March 18-21, 2008, Atlanta, GA. https://smartech.gatech.edu/bitstream/1853/20665/1/forest_trees_kinman.pdf

A description of a 5 year study on library services and resource usage, including a novel application of Tufte’s Sparklines http://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=0001OR

Kinman’s Application of Sparklines

Future Directions for Usage Data Analysis • Auditing or compliance with standards

• Non-text media (eBooks, podcasts, etc.)

• Non-text subjects (i.e. Museums, Art)

• More robust database analysis

• Development of user-centered statistical standards

• Develop standard measures and standard tests to help in evaluation

Thank you!

Comments:John McDonald

Libraries, Claremont University Consortiumjohn.mcdonald@libraries.claremont.edu