information studies introduction to webometrics: web-based ... · pdf fileintroduction to...
TRANSCRIPT
![Page 1: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/1.jpg)
Introduction to Webometrics:Web-based Methods for
Research Evaluation
Information Studies
Mike ThelwallStatistical Cybermetrics Research Group
University of Wolverhampton, UK
![Page 2: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/2.jpg)
Contents
□ What is webometrics?
□ Webometrics in traditional research evaluation
□ Webometrics for new research □ Webometrics for new research evaluations
□ Webometrics for social science
![Page 3: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/3.jpg)
1. What is webometrics?
![Page 4: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/4.jpg)
1. What is webometrics?
□ Webometrics is gathering data on the Web, and measuring aspects of the Web:
□ web sites
□ web pages
□ hyperlinks
□ web search engine results
□ YouTube video commenter networks
□ MySpace Friend networks
□ …for varied social science purposes
![Page 5: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/5.jpg)
New problems: Web-based phenomena
□ Webometrics can analyse online academic communication
□ Why do academic web sites interlink?
□ Which academic web sites interlink?□ Which academic web sites interlink?
□ What academic interlinking patterns exist?
□ Which web sites/groups/universities have the most online impact, and why?
![Page 6: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/6.jpg)
Old problems: Offline phenomena reflected online
□ Some offline phenomena have measurable online reflections
□ International communication
□ Inter-university collaboration□ Inter-university collaboration
□ University-business collaboration
□ The impact or spread of ideas
□ Public opinion about science
![Page 7: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/7.jpg)
Example: The online impact of research groups (NetReAct)
![Page 8: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/8.jpg)
Geopoliticalconnected
UK
Germany
Austria Switzerland
Belgium
Spain
France
NL
Example:Links betweenEU universities
Normalised linking, smallest countries removed
SwedenFinland
Norway
UK
Poland
Italy
NLEU universities
![Page 9: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/9.jpg)
International biofuels research network
![Page 10: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/10.jpg)
Data Gathering/Processing Tools
□ LexiURL Searcher – links, web text, YouTube, Flickr, Technorati
□ Submits thousands of queries to Live Search/Yahoo and summarises the results Search/Yahoo and summarises the results in standard ways
□ SocSciBot – links, web text
□ Web Crawler &
analyser
![Page 11: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/11.jpg)
2. Webometrics in traditional research evaluationresearch evaluation
![Page 12: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/12.jpg)
2. Webometrics in traditional research evaluation
□ Webometrics can supplement traditional citation impact non-traditional online impact
□ E.g., educational, discussion-based□ E.g., educational, discussion-based
□ It is typically weaker than citation data but is useful for research groups that have non-standard types of impacts
![Page 13: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/13.jpg)
The Integrated Online Impact Indicator (IOI)
□ Combines a range of online sources into one indicator
□ Google Scholar +
□ Google Books +□ Google Books +
□ Course reading lists +
□ Google Blogs +
□ PowerPoint presentations = IOI
□ OR select individual separate components
Invented by Kayvan Kousha
![Page 14: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/14.jpg)
New source 1: Google Scholar
□ Wider evidence of academic impact
□ Wider types of academic publications, some non-academic publications
□ Not reliable□ Not reliable
□ Coverage variable
□ Can’t be automatically queried
□ Free
![Page 15: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/15.jpg)
New source 2: Google Books
□ Books typically not indexed in WoS or Scopus
□ Relevant in book-based disciplines (arts, humanities, some social sciences)(arts, humanities, some social sciences)
□ Reliability unknown but probably not good
□ Coverage variable
□ Can be automatically queried
□ Free [Clifford Lynch]
![Page 16: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/16.jpg)
New source 3: Course reading lists
□ Evidence of educational impact
□ Can automatically construct queries to detect individual articles in online syllabusessyllabuses
□ Get results via advanced Google/Yahoo/Live Search queries
□ Works for most articles
□ Fails for short common article titles
![Page 17: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/17.jpg)
New source 4: Blogs
□ Evidence of impact on discussions
□ Educational impact, public dissemination evidence, academic impact in discursive subjects?impact in discursive subjects?
□ Not possible to automate in the largest database (Google Blogs)?
□ Not a well researched area
![Page 18: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/18.jpg)
New source 5: PowerPoint Presentations
□ Evidence of educational/scholarly impact
□ Especially relevant for discursive subjects?subjects?
□ Automated Live Search/Yahoo advanced queries
IOI = a*Scholar + b*PowerPoint + c*Blogs + d* Syllabus + e* Books
□
![Page 19: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/19.jpg)
3. Webometrics for newresearch evaluationsresearch evaluations
![Page 20: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/20.jpg)
3. Webometrics for new research evaluations
□ Some organisations produce non-academic research & need impact evaluation
□ Think tanks, government departments, □ Think tanks, government departments, charities
□ Target audience not academic
□ WoS/Scopus citations irrelevant (and almost non-existent)
□ May use LexisNexis media mentions
![Page 21: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/21.jpg)
Web citation analysis
□ Count mentions of report on the web
□ Evidence of wider public and media interest
□ Could focus on just blogs□ Could focus on just blogs
□ Can be automated (LexiURL Searcher)
□ Lots of Spam
□ Need manual checking and content analysis for the best results (expensive)
![Page 22: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/22.jpg)
Web citation analysis of online PDF and word documents
□ Evidence of “professional impact”
□ Academic documents, newsletters, white papers & the grey literature
□ Can be automated via Yahoo!/Live □ Can be automated via Yahoo!/Live Search document type-specific queries (filetype:pdf)
□ Variable quality documents
□ High quality documents can be very interesting
![Page 23: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/23.jpg)
NESTA Web Reports
□ National Endowment for Science, Technology and the Arts
□ Conducted twice-yearly for several years
□ Evaluate the online impact of the most □ Evaluate the online impact of the most recent 20 reports
□ Identify successful and unsuccessful reports
□ Identify successful and unsuccessful types of report
![Page 24: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/24.jpg)
![Page 25: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/25.jpg)
NESTA Research Citation Index
Citing document title and information TypeCited NESTA document
Legitimising RTI-policy: Market failure and systems failure - Two sides of one coin? -and beyond, Paper presented at the EAEPE Conference, Porto, 1-3 November 2007, Klaus Kubeczko & Matthias Weber
Conference paper
Demanding Innovation2007, Klaus Kubeczko & Matthias Weber paper Innovation
Public procurement and innovation -Resurrecting the demand side, Edler J, Georghiou L, (2007). Research Policy, 36(7), 949-963.
Journal article
Demanding Innovation
Innovation Nation, DIUS, March 2009Research Report
Demanding Innovation
Full index contains 100s of records. It is provided for browsing by NESTA
![Page 26: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/26.jpg)
4. Webometrics for social sciencescience
Large-scale quantitative web measurements for social science research goals
![Page 27: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/27.jpg)
Example 1: Blog searching
The graph isevidence that theissue started in theWest at this date
No other source of information willtell you that people were notthinking about the issue earlier!
= large scale quantitative= large scale quantitativeWeb measurements for social science research
Free at blogpulse.com
![Page 28: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/28.jpg)
YouTube debates for “Law Library Part III”
Generated usingLexiURL Searcher
red = happy replies, black = angry replies
![Page 29: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/29.jpg)
YouTube debates about Justin Bieber
![Page 30: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/30.jpg)
Summary
□ The web contains a wide variety of web and web 2.0 content posted by many different people in many different formats
□ Webometric methods can exploit this for □ Webometric methods can exploit this for evidence of a wide variety of types of impact
□ New clients and social scientists that we can help!
□ Happier old clients!?
![Page 31: Information Studies Introduction to Webometrics: Web-based ... · PDF fileIntroduction to Webometrics: Web-based Methods for Research Evaluation ... YouTube video commenter networks](https://reader034.vdocument.in/reader034/viewer/2022051320/5a9f64bc7f8b9a76178cc1bf/html5/thumbnails/31.jpg)
Books
□ Thelwall, M. (2009). Introduction to webometrics: Quantitative web research for the social sciences. New York: Morgan & Claypool.
□ Kousha, K., Thelwall, M. & Rezaie, S. (2010). Using the web for research evaluation: The
□ Kousha, K., Thelwall, M. & Rezaie, S. (2010). Using the web for research evaluation: The Integrated Online Impact indicator, Journal of Informetrics, 4(1), 124-135.
□ http://lexiurl.wlv.ac.uk http://webometrics.wlv.ac.uk