“The coolest thing to do with your data will be thought of by someone else”

Dave PatternLibrary Systems ManagerUniversity of Huddersfieldd.c.pattern@hud.ac.ukwww.daveyp.com/blog/

Graham StoneInformation Resources Manager

University of Huddersfieldg.stone@hud.ac.uk

@Graham_Stone

• …to improve existing services• …to gain insights into user behaviour• …to measure the impact of the library

– do the students who use the library the most get the highest grades?

Using Usage Data…

• User activity data– record of a user’s actions on a website or software

system or other relevant institutional service

• Attention data– record of what a user has viewed on a website or

software system or other relevant institutional service

http://bit.ly/g6S5wH

Defining Using UsageJISC Activity Data Programme

• Circulation Transactions– user A borrowed this book on 23/Jan/2011

• E-Resource Usage– user B logged in and accessed ScienceDirect 126

times during 2009

• Entry Stats– user C entered the library 12 times in Dec/2009

Defining Usage Datasome library examples...

5

Usage Data at Huddersfield

6

Keyword Cloudbased on keyword search data

Borrowing Suggestionsbased on circulation data

7

8

Borrowing Suggestionsbased on user’s recent loans

9

Course Level New Book Listsanalysis of course loans & Dewey

10

Search Suggestionsbased on keyword search data

11

Guided Keyword Searchesbased on keyword search data

12

Journal Suggestionsbased on link resolver logs

13

The Impact of Serendipity

14

Trends in Borrowingunique titles circ’ing per acad. year

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

80,000

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

unique titles (bib#s)

15

Trends in Borrowing# of items borrowed per acad. year

10

15

20

25

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

average number of items borrowed per academic year

17.10 18.92

16

Discussing Open DataJISC TILE Project meeting (Jun 2008)

• “The coolest thing to do with your data will be thought of by someone else.”– Rufus Pollack (Open Knowledge Foundation)

http://m.okfn.org/files/talks/xtech_2007/

Sharing DataWhy should I release my data?

• “Absolutely, couldn’t agree more. The point is not so much whether this statement might be true or not, so much as what it does to your thinking and planning if you decide to take it as an article of faith.”– Paul Walk (UKOLN) http://bit.ly/9ao3c4

Sharing DataWhy should I release my data?

19

Sharing Datausage data release (Dec 2008)

20

Sharing DataBA Multimedia Design

24

Non & Low Usage Project

25

Non & Low Usage Projectdigging deeper into data since 2005!

Non & Low Use Projectdigging deeper into data

27

Measuring Library Impactis there a link between usage & grade?

Non & Low Use ProjectResults

• Not a cause and effect relationship

• Never proven statistically significant

• Potential for collaboration on future projects

http://www.flickr.com/photos/atoach/3344411469/

JISC Activity Data Call

Library Impact Data Project

To prove the hypothesis that…

“There is a statistically significant correlation across a number of

universities between library activity data and student attainment”

Project reportshttp://library.hud.ac.uk/blogs/projects/lidp/

• Themed posts– The Project Plan– Hypothesis– Users– Benefits– Technical and Standards– Licensing & reuse of software

and data– Wins and fails (lessons along

the way)– Final post

Data requirements

• For each student who graduated in a given year, the following data was required:– Final grade achieved– Number of books borrowed– Number of times e-resources were accessed – Number of times each student entered the library,

e.g. via a turnstile system that requires identity card access

– School/Faculty

Legal issues

• Consultation with JISC Legal, University legal officer and data protection officer

• Ensured that any identifying information is excluded before it is handled for analysis

• Excluded any small courses to prevent identification of individuals e.g. where a course has less than 35 students and/or fewer than 5 of a specific degree level

• Received guidance from the Using OpenURL Activity Data

Data issues

• Anticipated that there may be problems in getting enough data to make the project viable– Potential partners were asked to confirm that they could

provide at least 2 of the 3 measures of usage as well as student grades

– Huddersfield has provided definitions on the data required and the form the data can be accepted in

• Some partners have already run into some issues with data collection, but it is felt that there is still enough information to prove the hypothesis one way or another

Can we prove the hypothesis?

• Not quite!

• Due to the data not being continuous, a correlation cannot be calculated

http://www.flickr.com/photos/26015375@N06/3715306069/

Further statistical tests

• Running a Kruskal-Wallis test– to indicate whether there is a

difference between values e.g. between levels of e-resource usage across degree results

– THEN we analyse the data visually to check which variables to compare

What we think we can prove

• That the relationship and variance means that you can believe what you see

• And you can believe it across a range of data, e.g. subjects

• So library usage does impact on students attainment

70

60

45

30

83

65

41

27

0

20

40

60

80

100

1 2:1 2:2 3

MetaLib logins books borrowed

Remember the disclaimer!

Not a cause and effect relationship

Library Impact Data Projectbook loans inc. renewals (2009/10)

First class honours Upper second class honours Lower second class honours Third class honours Pass - degree awarded without honours

0

50

100

150

200

250

300

350

400

218230

170

124

105

249242

180

135

151

279

236

156

126

100

2007/08 2008/09 2009/10

41

Library Impact Data Projectbook loans & Athens (2009/10)

42

Library Impact Data Projectlibrary PC logins & visits (2009/10)

Linking back to non/low usage

• Our research shows that for books and e-resource usage, there appears to be a statistical significance across all partner libraries

• If we know that there is a link between usage and attainment– We can link this back to non/low usage

44

Measuring Library Impact2008/9 – library visits

15.5% of students who gaineda 1st never visited the library

34% of students who gaineda 3rd never visited the library

45

Measuring Library Impact2008/9 – MetaLib usage

70% of those who gained a3rd logged in to e-resources

20 times or less over 3 years

10.5% of students who gained a1st logged in more than 180 times

46

Measuring Library Impact2008/9 – book loans

15% of students who gaineda 1st never borrowed a book

34% of students who gaineda 3rd never borrowed a book

Profiling non/low users

• Flesh out themes from the focus groups– to advise on areas to work on

• Check the amount and type of contact subject teams have had with the specific courses– to compare library teaching hours to attainment

• Baseline questionnaire or exercise for new students– To establish the level of information literacy skills for new

students• Target our users by concentrating staff resources at the right

point

Next steps for the project

• Pull out any themes from the focus groups• Release the data on an Open Data Commons Licence• Release a toolkit to help others benchmark their data

Further work

• Do cuts to the information budget mean that attainment will fall?

• Can we add more value by better use of resources?• What about gender and socio-economic background?• Can we link books borrowed on reading lists to attainment• What about VLE use?• Interest from ACRL (USA), University of Wollongong

(Australia), the Netherlands and Denmark

Acknowledgements

• Dave Pattern and Bryony Ramsden

• Phil Adams, Leo Appleton, Iain Baird, Polly Dawes, Regina Ferguson, Pia Krogh, Marie Letzgus, Dominic Marsh, Habby Matharoo, Kate Newell, Sarah Robbins, Paul Stainthorp

Thank you

• http://library.hud.ac.uk/blogs/projects/lidp/

• http://eprints.hud.ac.uk/10949/

This work is licensed under a Creative Commons Attribution 3.0 Unported License