what is bibliometrics and how does it work?
TRANSCRIPT
Maynooth University
What is Bibliometrics and how does it work?Ciaran Quinn, Research Support Librarian, Maynooth University6th May 2015
17/04/2015
Maynooth University
What is Bibliometrics?Statistical analysis of bibliographic data by
counting citations ◦ Measures patterns of authorship , publication and
the use of the literature. ◦ Usually Citation analysis of research outputs &
publications◦ Assess Research impact of individuals, groups,
institutions◦ Measuring by Author (H Index), Article (Plos), or
Publication (Journal Impact Factor)◦ Measure of Output not Quality (Quantitative Not
Qualitative !) Other measures could include funding received, number
of patents, awards granted, or qualitative measures such as peer review
17/04/2015
Maynooth University
Why use Bibliometrics? Measure of the Impact of your Research
◦ Your Publishing activity !◦ CV, Promotion, Tenure, Grants, Feedback to funding
bodies/Industry/Public◦ Showcase Individual/Group/Institutional Research
Identify Areas of Research Strengths/Weaknesses ◦ Inform Research Priorities
Identify highest impact or top performing Journals in a Subject Area◦ Where to Publish, learning about a particular subject
area, identify emerging areas of research Identify the top researchers in a subject area
◦ Collaborations/Competitors◦ Recruitment◦ Learning about a subject area
17/04/2015
Maynooth University
Examples of Metrics Scholarly Output: How many articles by an individual, Research
Group, University Citation counts (to measure the impact of an article or author)
No one comprehensive source for all articles !
Field-weighted cited impact (SciVal) Outputs in top percentiles (SciVal)
Journals in top 10% Good for early career researchers to be in the top journals !
Collaboration & Potential Collaboration metrics (SciVal) H-Index:
available from a number of sources
Journal Impact Factor (Web of Science) Compare Journals: SJR, SNIP, IPP (Scopus) ScImago Journal & Country Rank: SJR (uses Scopus Data)
Weighted by the Prestige of a Journal
Google Scholar Citations (Citations/H Index) Publish or Perish (Author Impact, Journal Impact) Good for unpublished material & books
17/04/2015
Maynooth University
Citation Tracking Tools Key Tools
Web of Science/Incites: ◦ Arts & Humanities, Social Sciences & Science
Citation Indexes (Thomson Reuters)Scopus/SciVal:
◦ Citation Overview/ Tracker, Analyse Results, Author Evaluator (Elsevier)
◦ Also uses Mendeley Readership Stats !SciMago: Journal & Country Rank (Elsevier)Publish or Perish (Harzing)Google Scholar Citations
17/04/2015
Maynooth University
Other Tools (More Limited !)
PubMed (Cited in PubMed Central) EBSCO: Find Citing Articles (Cited by EBSCO) IEEE Xplore (Electrical Eng & Computer Science)Cited by
IEEE ACM Digital Library (Cited by) JSTOR: (Items citing this item) Anthro Source (Citation alerts) SciFinder (Get References & Get Citing Options) Cite Seer citation statistics
◦ Computer & Information Science MathsSciNet: Author & Publications Citations Altmetics tools: Look at non traditional metrics at article
level◦ Impact Story, Altmetrics.com, Mendeley (Elsevier) Plum Analytics
(EBSCO)
17/04/2015
Maynooth University
Other Journal Rankings listing
Washington & Lee University School of Law: Law Library Law Journals: Submissions and Ranking, 2006 - 2013 :Journal Citations
http://lawlib.wlu.edu/LJ/Association of Business Schools
http://www.bizschooljournals.com/SciMago http://
www.scimagojr.com/journalrank.php?area=1400
Journal Quality List (Harzing) http://www.harzing.com/jql.htm
17/04/2015
Maynooth University
Issues & Limitations (1) Sciences (quantitative) Vs Humanities (qualitative) :
Do Bibliometrics suit sciences better? Citation rates in Humanities much lower (Social Sciences fare better)
Citation patterns can differ greatly between disciplines. Need to compare like with like !
Some disciplines don’t publish as much in Journals Computer Science, Electrical Engineering, Arts, Humanities, Social
Sciences
Only published Literature included Policy papers, Professional Journals, Working papers, Monographs,
Reports ,Dissertations
Citation does not have to be positive! Citation tools don’t index all research areas or publications .
Different tools/different datasets = different results
17/04/2015
Maynooth University
Issues & Limitations (2) Manipulation of the system!
◦ Self citing, citing colleagues, splitting outputs into many articles, participation in multi authored publications
Editors can manipulate their journal’s ranking by asking authors to include more citations of other articles from that journal (Coercive citation!)
Some Publishers overly concerned with increased Impact Factor ◦ Rejecting quality articles !
Difficult for early career researchers to show their output, metrics take time◦ Reflect poorly against more experienced researchers.
Measures Impact (number of citations) not the quality of the work !◦ Quantitative not qualitative
Academic freedom? Where you publish, What you publish, How often you publish◦ Value of research vs getting cited !
◦ Shorter research cycles !
◦ Is curiosity driven research too narrow for journals to be interested?
◦ Increase in shorter articles to “Letters” Journals (Biology, Physics)17/04/2015
Maynooth University
Recent Articles“Bibliometrics: The Leiden Manifesto for research
metrics”: Diana Hicks et al,(2015)http://www.nature.com/news/bibliometrics-the-leiden-manifesto-for-research-metrics-1.17351
“The focus on bibliometrics makes papers less useful”
Forcing research to fit the mould of high-impact journals weakens it. Hiring decisions should be based on merit, not impact factor, says Reinhard Werner. (2015)
http://www.nature.com/news/the-focus-on-bibliometrics-makes-papers-less-useful-1.16706
17/04/2015
Maynooth University
Key points Compare like with like
◦ Similar Research Areas◦ Stage of academic career◦ Similar Journals (Discipline)◦ Similar size institutions
Don’t just use one tool ! ◦ Coverage varies in content, depth, discipline
Does your discipline publish in journals and are those journals included in the Datasets being used to generate metrics?
Data needs to be looked at in context ! Use a variety of metrics and other qualitative information where appropriate
17/04/2015
Maynooth University
What Metrics Should I use?
What question am I trying to achieve/answer?
What am I evaluating/showcasing?
How will I recognise good performance?
Which metrics will help me?
17/04/2015
Maynooth University
Practical Session: Creating a Researcher Profile; Researcher Impact; Citation
Tracking; Journal Impact
Ciaran Quinn, Research Support Librarian, Maynooth University.6th May 2015
17/04/2015
Maynooth University
Researcher Identity & Publication Profiles
Why do it? Avoid Author & Institutional Disambiguation
◦ Be consistent in the variation of your name (& Institutional name) that you use
◦ Makes sure your work is correctly attributed to you◦ Groups your name variants and different institutions you may
have published in together Improves Visibility = Being Found & Read=More Citations
◦ Helps Researchers, Students, Funding Agencies, Potential Collaborators find your work
Helps you track and measure the impact of your research Easier for you to create a list of your publications and
sort them by impact Monitor your Researcher Identity and keep it up to date !
17/04/2015
Maynooth University
What Tools are available? Institutional
◦ RIS (Research Information System) Maynooth◦ RPD (Researcher Profile Directory)
http://apps.maynoothuniversity.ie/rpd/Researcher ID (Thomson Reuters)Scopus Author Identifier (Elsevier)Open Researcher & Contributor ID (ORCID)Google Scholar Citations Peer Networks (Academia.edu, ResearchGate.net
, Mendeley.com, Social Science Research Network (SSRN.com), LinkedIn.com
Utilise social bookmarking with Mendeley, Zotero or CiteULike.org
17/04/2015
Maynooth University
Researcher ImpactKeeping track of your research
◦Check your H Index◦Set up Citation Alerts: WOS, Scopus,
Google Citations & others ◦Research/Social Networking sites &
Altmetrics to see who is view/downloading or discussing your work
17/04/2015
Maynooth University
Improving your Research ImpactIncrease your visibility
◦Set up a Researcher ID (WOS), Check & update Your Scopus ID
Export them to Orcid
◦Google Scholar Citations◦Social Media/Social Bookmarking/Blogging◦ Institutional Repositories◦Make yourself findable by linking up you
Research title, Article title and Keywords◦Posters/Presentations at Conferences◦Collaborate with other academics◦Submit articles to the highest ranking journals
17/04/2015
Maynooth University
Citation Tracking using Alerts
ScopusWOS: Citation Indexes
◦ (Humanities, Social Sciences, Sciences, Conferences, Books)
Google Scholar
17/04/2015
Maynooth University
Other Citation SourcesERIC (Cited by)High Wire (Cited by High Wire article)HeinOnline (Cited by) JSTOR (Items citing this item)ProQuest (Cited by other articles in the
database)Science Direct (Citing articles from Scopus)Springer Link (Cited by)SSRN (Citations tab)Wiley (Cited by) Ingenta Connect Institute of Physics: IOP (All citing articles)
17/04/2015
Maynooth University
Why are Journal Rankings Important? Journals are ranked by how frequently recently published
papers from a particular journal are cited Prestige of publishing in a particular Journal !
◦ Bur does it measure the academic value/research quality of an article!
However it does have an impact on the reputation of a researcher and therefore its important for careers◦ Used by funding agencies and academic institutions◦ 36% of top cited articles in top 5% of Journals
Gradually declining
Metrics can be gamed◦ Journals may decide to publish on certain topics or certain article
types (such as reviews) to maximise their JIF San Francisco Declaration on Research Assessment 2012
IF creates biases and inaccuracies when appraising scientific research. It also states that the impact factor is not to be used as a substitute "measure of the quality of individual research articles, or in hiring, promotion, or funding decisions”
17/04/2015
Maynooth University
Journal Rankings/MetricsJournal Citation Reports (includes
Journal Impact Factor): Thomson Reuters (2 & 5 years)
Journal Compare/Analyzer: Scopus (Elsevier)(3years)
SciMago Journal & Country Ranking (Elsevier)(3 years)
Google Scholar Metrics (Top Publications, 5 years)
Publish Or Perish17/04/2015
Maynooth University
Other Journal RankingsCreated Citations e.g. Legal & BusinessWashington & Lee University School of
Law: Law Library Law Journals: Submissions and Ranking, 2006 - 2013 :Journal Citations
http://lawlib.wlu.edu/LJ/Association of Business Schools
http://www.bizschooljournals.com/SciMago http://
www.scimagojr.com/journalrank.php?area=1400
Journal Quality List (Harzing) http://www.harzing.com/jql.htm
17/04/2015
Maynooth University
Eigenfactor.Org◦http://eigenfactor.org/index.php
Centre for advanced studies in Management and Economics (List of Journal Ranking Tools)◦http://
www.cefage.uevora.pt/en/links/revistas_cientificas_rankings
Journal Metrics.com (SciMago)◦http://www.journalmetrics.com/about
-journal-metrics.php17/04/2015
Maynooth University
Benchmarking individual/institutional ResearchTools
◦Scival (Elsevier) SciVal offers quick, easy access to the
research performance of 4,600 research institutions and 220 countries worldwide. Overviews, Benchmarking, Collaborations
◦Incites (Thomson Reuters) Incites is a tool that allows you to analyze
institutional productivity and benchmark your output against peers worldwide.
17/04/2015
Maynooth University
Author Name Look up your own author name (or someone you
know) on Scopus, Web of Science and Google Scholar◦ Are they there? Does the data adequately represent
them?◦ Are there many differences between the data sources?
Number of documents, H Index
◦ How many name variants are there?◦ Are there many errors?
Set up a profile on Researcher ID (Wos) and Google Citations. Check your profile on Scopus. Export your profile to Orcid.
Are there corrections that will need to be made?◦ Name variants/Correct Documents assigned
17/04/2015
Maynooth University
Citation AlertsSearch for your articles and set
up a citation alert ◦Do this in WOS, Scopus and Google
Scholar
17/04/2015
Maynooth University
Journal ImpactLook at the impact of an
important journal in you field (use WOS & Scopus)◦Is it included in the datasets?◦If not does that surprise you?
Identify the top Journal in your research area
17/04/2015
Maynooth University
The EndQuestions?
Contact: [email protected]
Research Support Blog: http://ciarnthelibrarian.blogspot.ie/
17/04/2015