Slides from Humanities on the Web: Is it working?Date: Thursday, 19 March 2009, 10-4Location: Oxford University, Oxford, UKWebcast URL: http://webcast.oii.ox.ac.uk/?view=Webcast&ID=20090319_274Slide URL: http://www.slideshare.net/etmeyer/TIDSR

Morning Event:10:00 – 11:15: Toolkits for e-Humanities

Overview of Phase 1 Digitisation Projects (Dr. Kathryn Eccles)Quantitative Tools (Prof. Mike Thelwall & Dr. Kathryn Eccles)Qualitative Tools (Dr. Eric Meyer & Dr. Kathryn Eccles)Survey Data (Christine Madsen & Dr. Eric Meyer)Discussion and questions

Full details: http://www.oii.ox.ac.uk/events/details.cfm?id=238

Toolkit for the Impact of Digitised Scholarly Resources: Introduction to JISC Phase One Digitisation projects

Kathryn Eccles

Oxford Internet InstituteUniversity of Oxford

Project 1 – Online Historical Population Reports (OHPR/Histpop)

Project 1 – Online Historical Population Reports (OHPR/Histpop)

Project 1 – Online Historical Population Reports (Histpop)

Project 2 – British Library 19th Century Newspapers

Project 2 – British Library 19th Century Newspapers

Project 2 – British Library 19th Century Newspapers

Project 2 – British Library 19th Century Newspapers

Project 3 – British Library Archival Sounds

Project 3 – British Library Archival Sounds

Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834

Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834

Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834

Project 4 – British Official Publications Collaborative Reader Information Service (BOPCRIS): 18th Century Official Parliamentary Publications Portal 1688-1834

Project 5 – Wellcome Medical Journals: the backfiles project

Project 5 – Wellcome Medical Journals: the backfiles project

Project 5 – Wellcome Medical Journals: the backfiles project

Find out more:

Find out more at the Project websites:

1. Histpop: http://www.histpop.org

2. British Library C19th Newspapers: http://www.bl.uk/reshelp/findhelprestype/news/newspdigproj/paperdigit.html

3. British Library Archival Sounds: http://sounds.bl.uk/

4. BOPCRIS: http://www.parl18c.soton.ac.uk/parl18c/digbib/home

5. Medical backfiles: http://library.wellcome.ac.uk/backfiles/

Find out more:

More about the JISC Digitisation Programme:

http://www.jisc.ac.uk/whatwedo/programmes/digitisation.aspx

Visit the toolkit at:

http://microsites.oii.ox.ac.uk/tidsr/

Oxford Internet InstituteUniversity of Oxford

Kathryn Eccleskathryn.eccles@oii.ox.ac.uk

http://www.oii.ox.ac.uk/people/faculty.cfm?id=138

Toolkit for the Impact of Digitised Scholarly Resources:Quantitative Methods

Dr Kathryn Eccles

Oxford Internet Institute, University of Oxford

&

Professor Mike Thelwall

University of Wolverhampton

Overview

Quantitative Methods used in the Toolkit for the Impact of Digitised Scholarly Resources:

• Webometrics

• (Google) Analytics

• Log file analysis

• Scientometrics / bibliometrics

Webometrics

What is webometrics?

• Gathering, processing and analysing large scale data from the web (web pages, hyperlinks, blogs, Web 2.0) for many purposes that include online communication, although primarily for scientific communication

What can webometrics tell us about the Impact of Digitised Scholarly Resources?

• Counting pages that link to scholarly resource archives can indicate the impact of the archives

• More links suggests more users and higher impact• Assessing the origins of links gives insights into users of the scholarly resources by finding out why the links were created

• E.g., universities, schools, bloggers, countries• Comparing the results with similar archives

• Indicates relative impact & potentially missed user groups

Webometrics

Using webometrics in the TIDSR project

• Comparator sites• Running the analysis – problems• Results:

– Histpop performs well against its comparator projects

Webometrics

– British Library C19th Newspapers records a large number of links for an information page

– British Library Archival Sounds performs well; contrasts heavily with British Library Sound Archive

– BOPCRIS C18th Parliamentary Papers records fewer links than to BOPCRIS homepage, but higher than to its commercial partner at ProQuest

– Wellcome Medical Backfiles project page records strong links, links to PubMed for WMB material impossible to trace

Analytics

What are (Google) analytics?

• Data about a web site’s visitors typically derived from tracking code hidden in each of the web site’s pages

• Google Analytics provides a free service and extensive summary statistics about visitors

• Commercial analytics can provide more in-depth and customisable statistics – probably most useful for large-scale commercial websites

What can analytics tell us?

Google Analytics can be extremely effective in providing information about (Fang, 2007) :

• The usage of a website

• Visitor behaviors

• The efficiency of the website's menu system

• Suggestions for improving user experiences

• The most effective redesign of the website

•The geographic origins of visitors

•Search queries sending visitors to the site

Log File analysis

What is Log File Analysis?

• (Web server) log files are simple plain text files that record who accesses the pages and other resources in a web site

• Each access is recorded on a separate line of the file• Who accessed each page is recorded by IP address

• Log file analysis programs read in the raw log files and produce useful summary statistics (similar to analytics), e.g.

• Daily or hourly number of visitors• Most popular pages• Average time on site of visitors• Search engine queries driving visitors to site• Geographic origin of visitors (from decoded IP addresses)

Log File analysis

Using log file analysis in the TIDSR project

• Collecting web statistics from log file dataComparing: - What kind of data was collected - How often this was consulted

• Collecting raw log files

Looking at:

- What search terms were used to find the resource

- The top referrer sites

- Access statistics

Log File Analysis

• Top Search Phrases: Histpop

Scientometrics/Bibliometrics

Scientometrics/Bibliometrics

• Bibliometrics are typically measurements of document collections and are traditionally mainly derived from scientific databases• A common (partial) indicator of the importance of a document is how often it has been cited by other documents• Often-cited documents tend to be thought of as more valuable to the scientific community• Bibliometrics are less relevant outside science in the humanities and social sciences and are not reliable for individual articles• Bibliometric indicators can help to evaluate the impact of a collection of documents

Scientometrics/Bibliometrics

How useful were Scientometrics/Bibliometrics in the TIDSR project?

• Google Scholar, Scopus, ISI Web of Knowledge/Web of Science• Limited time scale• Examples from projects (and many were about the projects, rather than links to project materials)

• Histpop: 11• BL 19th Century Newspapers: 4• BL Archival Sounds: 9• BOPCRIS: 8• Medical Journals: 17

• Importance of trackable citation styles

For more information, visit the toolkit at:

http://microsites.oii.ox.ac.uk/tidsr/

Toolkit for the Impact of Digitised Scholarly Resources:Qualitative Methods

Dr Eric Meyer

&

Dr Kathryn Eccles

Oxford Internet Institute, University of Oxford

Overview

Qualitative Methods used in the Toolkit for the Impact of Digitised Scholarly Resources:

• Interviews

• Focus groups

• User feedback

• Referrer analysis

Interviews

Interviews

• Key project personnel

• Related but non included institutional personnel

• End-user communities

• Subject specialists

• Other stakeholders

• Funding personnel

Interviews

Project impacts revealed through interviews

• Quality of undergraduate dissertation work improved by early contact with digitised primary sources

‘Histpop made it possible to do a completely different project’ • Type of research projects being presented at conferences

increasingly quantitative

‘You do feel the ground shifting...’ • New possibilities for serendipitous research

‘What I’ve found most enjoyable... is the serendipity’

Focus groups

Focus groups

Themes we covered:

• General attitude to and knowledge of digital resources

• Methods of searching for digital resources

• Citation of digital resources

• Issues of trust in discovering and using digital resources

• Training and support in using digital resources

• Resource specific questions – findability and usability  

User feedback

Using User feedback within your own resource:

• Keep a record of all the correspondence you receive from users

• Periodically review the content of your user feedback (classify each response as negative, positive, both or neutral

• Use contact with users to recruit focus group and survey participants

• Use contact with users to gather information about how your resources is being used (Distance learning courses, undergraduate dissertations)

• Use this extra feedback to enhance the impact of the resource on these communities

User feedback

User feedback in the TIDSR project

• Projects were reticent about providing email content from users

• We were only able to gather anecdotal evidence from most resources

User feedback from Histpop:

• We were allowed to send an email to be forwarded to users who had contacted Histpop

• This allowed us to contact a range of users and ask them detailed questions about the resource

• This yielded further information about the ways in which the resource was being used, particularly in the teaching and learning environments

Referrer analysis

Referrer Analysis

• Similar to the quantitative webometrics in approach, but requires a much more qualitative look at the resulting pages to try to find educational referrers (such as libraries and course reading lists)

For more information, visit the toolkit at:

http://microsites.oii.ox.ac.uk/tidsr/

TIDSR Survey Data

Eric MeyerChristine Madsen

OII19 March 2008

• Ave. time spent filling out survey: 17.5 minutes

• 83% completed the survey in under 20 min

• 714 started

• 550 completed (77%)

• 80% agreed to continue on to further questions

Demographics

Demographics

Average Age 45.3

Age Range 20-83

Degrees & Field

7%

12%

16%

17%

31%

34%

45%

47%

51%

54%

58%

62%

71%

77%

83%

13%

11%

9%

30%

18%

29%

32%

38%

27%

24%

32%

36%

56%

50%

48%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

Cornell Animal Sounds

Sciper

Histpop

Chronicling America

Fine Rolls

Wellcome Medical Journal Backfiles

Historical Directories

Internet Lib of Early Journals

BOPCRIS

Archival Sounds

Imperial War Museum

Old Bailey Online

British Periodicals

British Library Newspapers

House of Commons Parliamentary Papers

Non-UK Awareness

UK Awareness

Awareness of Resource by Country

Have you ever published a piece based on your work in this collection?

If so, how did you cite the collection?

How did you find this resource?

How did you find this resource?

Histpop BOPCRIS BL News BL Sounds Med Backfiles

Google 33% 33% 25% 19% 37%

Prof society, discussion list or conference

25% 22% 26% 43% 27%

Library 4% 22% 32% 10% 27%

Colleague 25% 18% 11% 24% 6%

Other 13% 4% 7% 5% 2%

Student 0% 0% 0% 0% 0%

Campus IT 0% 0% 0% 0% 0%

How do you use this resource?

How do you use this resource?

Histpop BOPCRIS BL News BL Sounds Med Backfiles

Online reference source

78% 81% 74% 66% 78%

For personal interests

35% 49% 50% 46% 27%

Download to use offline

26% 36% 40% 29% 57%

Teaching resource

30% 30% 29% 39% 27%

Manual analysis 17% 32% 38% 10% 39%Find materials to consult in person

4% 23% 18% 12% 39%

Computational analysis

30% 6% 4% 2% 10%

Reuse, remix or edit

17% 11% 9% 7% 0%

Other 4% 0% 3% 5% 6%

Attitudes towards Digitisation

For more information, visit the toolkit at:

http://microsites.oii.ox.ac.uk/tidsr/