google scholar as a cybermetric tool
DESCRIPTION
Google Scholar as a cybermetric tool. Alastair G Smith Victoria University of Wellington New Zealand [email protected]. Overview. Google Scholar as a cybermetric tool - PowerPoint PPT PresentationTRANSCRIPT
Google Scholar as a cybermetric tool Alastair G SmithVictoria University of Wellington New Zealand
Overview
Google Scholar as a cybermetric toolUsed to compare web citation rates with results of a research assessment exercise: New Zealand’s Performance Based Research Fund (PBRF).
Cybermetrics
Quantitative analysis of published research on the InternetEmulates techniques used by bibliometrics in the conventional publishing environment
Tools for cybermetric work
General purpose search engines, e.g. Alta Vista Google
Specialised cybermetric crawler, e.g. Wolverhampton Academic Web Link
Database (http://cybermetrics.wlv.ac.uk/database)
General purpose search engines
Search algorithms not transparentPriority is to provide satisfactory search, not exact resultCover all of web, not just research material
Specialised cybermetric crawler
Search algorithm publicly documentedOriented to cybermetric purposesFocussed on specific research institutions
Google Scholar
Indexes research oriented material on the Internet (though selection criteria not published)Includes citation informationCitations include print material referenced in Internet documentsPotential as cybermetric tool?
PBRF research assessment
Research assessment exercise: NZ Tertiary institutions 2003/4Assessed research outputs from staff Institution ratings include: Total output Quality score: average output per staff member
Ratings determine grants from Performance Based Research FundBenchmark to compare with cybermetric study?
Specialised crawler study
Previous study (Smith & Thelwall 2005) Compared links to NZ university web sites with PBRF rankings.Moderate correlation: Link count/staff cf PBRF quality score
Issues: NZ-NZ links, ignores international linkage Includes non-research material
Google Scholar study
Searched Google Scholar for research originating from the 8 NZ universitiesCitation counts for this material extractedCompared with PBRF ratings
Methodology issues with Google Scholar
Problems identifying output of institutionNo total citation count provided
Identifying institution
No “institution” field Domain name search produces false drops (e.g. mirrored research ), misses research hosted at other institutions (e.g. paper presented at external conference)Searched on words in name of institution, and location e.g. "canterbury university" OR "university of canterbury" zealand OR christchurch OR ilam
Google Scholar result screen
Obtaining citation count
Total citation count not displayedOnly first 1000 hits obtainable (but citation counts appear to be used in ranking)Extracted individual citation counts with macroAssumed citation counts for first 1000 hits a good approximation for total citation count.
Results: Citations vs PBRF output
0
1000
2000
3000
4000
5000
6000
7000
0 10000 20000 30000 40000 50000
Google scholar citations
To
tal P
BR
F o
utp
ut
Results: Hits vs PBRF output
0
1000
2000
3000
4000
5000
6000
0 10,000 20,000 30,000 40,000 50,000 60,000
Google Scholar hits
To
tal
PB
RF
ou
tpu
t
Comparison of cybermetric tools
Coverage Identification of institutions
Citation count
Transparency
Google Scholar Research on Web
By keyword Individual, cannot display all hits
Little documentation of algorithm, selection of sources
Web of Knowledge
Core journals (some digital)
Specific field For individual items
Sources documented
Scopus Core journals + Web sites (from Scirus)
Specific field “Citation tracker” only for authors
Sources documented
Wolverhampton Crawler
Specific university web sites
By domain Link counts
Sources documented
Pros and Cons of using Google Scholar for cybermetrics
Pros: Good coverage of research on Web Accessible and simple
Cons Not transparent Need to use ad hoc search for
institution
Institutional repositories
Could provide good target for specialised cybermetric crawler, if: Standardised formats for citations etc Standard selection criteria Accessible to crawlers
Summary
Google Scholar provides a useful tool for “quick and dirty” cybermetric workResults have some correlation with a general research assessmentInstitutional repositories may provide a useful way of evaluating institutions