using google scholar in research evaluation of social science research
DESCRIPTION
Presentation for the STI/ENID conference 2014, september, Leiden, The NetherlandsTRANSCRIPT
![Page 1: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/1.jpg)
Using Google Scholar in research evaluation of social science
programsAd Prins
Rodrigo CostasThed van Leeuwen
Paul Wouters
![Page 2: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/2.jpg)
Bibliometrics for Social Science Programs?
• Coverage in WoS hampers bibliometric analyses
• Deans of social science faculties in the Netherlands: Experiment for a bibliometric study in two research assessments in SSH
• Google Scholar (GS) based for two fields
– Education and Pedagogical Sciences (Ed/Ped)
– Anthropology
September 3, 2014 2Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
![Page 3: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/3.jpg)
Selection of Data
• Ed/Ped: 13 programs of six universities, 2006-2011
• Anthropology: 5 programs of 5 universities, 2004-2012
• Selection of publications
– Ed/Ped: directors send in 10 publications
– Anthr.: 5-10 most cited publications depending on size of program
September 3, 2014 3Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
![Page 4: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/4.jpg)
Google Scholar Data Retrieval and Validity test
2nd
order data
1st order data
4September 3, 2014Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
![Page 5: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/5.jpg)
Data quality
Total number of verifiable citations
Percentage of total found citations
Ed/Ped 22887 89,8%
Anthropology 8092 89,7%
September 3, 2014 5Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
Verifiable (2nd order) citation data in GS:
Proper internet address (URL) to journal, academic publisher, university library, acknowledged repository (Mendeley, SSRN, Academia.edu etc), academic societies etc.
![Page 6: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/6.jpg)
Citing sources in Scholar
September 3, 2014 6Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
Source Anthro-pology #
Anthro-pology %
Ed/Ped # Ed/Ped %
Academic sources
4573 56.5% 14470 63.2%
University Libraries
1677 20.7% 4573 20.0%
Repositoriesother than ULs
616 7.6% 1236 5.4%
Other Sources 327 4.0% 1085 4.7%
Unknown 899 11.1% 1523 6.7%
Total 8092 100.0% 22887 100.0%
56% to 63% of total volume of citations is found in academic sources (scientific journals and academic publishers)About 20% of the citations are located in University Libraries
![Page 7: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/7.jpg)
Cited documents
September 3, 2014 7Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
![Page 8: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/8.jpg)
Representation in WoS
September 3, 2014 8Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
![Page 9: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/9.jpg)
Comparing GS and WoS (1)
September 3, 2014 9Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
Citations in GS Citations in GS, Academic
sources
Citations in WoS
Anthropology 8092 4573 1097
Education & Pedagogical Sciences
22887 14470 8870
• Information in GS exceeds that of WoS even when limited to academic sources (scientific journals and academic publishers)
![Page 10: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/10.jpg)
Comparing GS and WoS (2)
September 3, 2014 10Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
Anthropology: programs differ in benefitting from using GS
![Page 11: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/11.jpg)
Comparing GS and WoS (3)
September 3, 2014 11Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
Ed/Ped:Most programs have similar results, but some don’t
![Page 12: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/12.jpg)
Issues in using GS
• Workload and important data limitations
• Possibilities for field normalization
• What is a citation? Shifting definition.
• Choosing a proper selection base
• The place of bibliometry in research evaluations
September 3, 2014 12Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
![Page 13: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/13.jpg)
Conclusions
• Google Scholar is a valuable resource:
– The amount of information in Google Scholar is much larger than in WoS: 2,5 to 8 times
– Social science programs may benefit
• But in using Google Scholar…
– Data reliability control and access are an important issue
– Only small/medium-scale studies are currently possible
– New forms for field normalization are necessary
– How much is Google Scholar indexing from the total output?
September 3, 2014 13Using Scholar in Research Evaluation
STI/ENID 2014, Leiden
![Page 14: Using google scholar in research evaluation of social science research](https://reader034.vdocument.in/reader034/viewer/2022042816/55979cde1a28abc8488b45cb/html5/thumbnails/14.jpg)
Ad PrinsSupport in ResearchManagement
www.adprins.nl
September 3, 2014Using Scholar in Research Evaluation
STI/ENID 2014, Leiden14
Rodrigo CostasThed van Leeuwen
Paul WoutersCentre for Science and Technology Studies, Leiden
www.cwts.nl