For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

CBMR-SciComm

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 1

Altmetrics

Assessing research impact beyond citations

Daniel NoesgaardCenter for Basic Metabolic ResearchScience Communication

What is Impact?

CBMR-SciComm

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 2

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

What is impact and how do we measure it?

Classical assessment of research

• Results of research is published as article in journal• No of citations

Journal impact factor

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 3

CBMR-SciComm

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

The Journal Impact Factor

Definition :

The average number of citations received per paper published in that journal during the two preceding years

The IF for 2012 is based on citations made in 2012 to papers published in 2010 and 2011.

Formula:

•Eugene Garfield, 1955•Originally created as a tool to help librarians select which journals to purchase•Calculated and published by Thomson Reuters

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 4

CBMR-SciComm

CBMR-SciComm

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 5

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

DORA - Declaration On Research Assessment

• Annual meeting - The American Society for Cell Biology (ASCB) December 2012, San Francisco• “Putting science into the assessment of research”1

Problems with Journal Impact Factor• Citation distributions highly skewed• Can be manipulated by editorial policy• Calculations are neither transparent, publically available

nor reproducibleExamples

• Editors requests further references2

• Attempts to reproduce IF consistently failed3

• 89% of citations in Nature generated by just 25% of papers4

1 DORA, http://am.ascb.org/dora/2 Editorial, PLoS Med 3(6): e2913 Rossner et al, Journal of Cell Biology 179 (6): 1091–2. 2 Editorial, Nature 435, 1003-1004

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 6

CBMR-SciComm

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

DORA (continued)

Signed by editors

Elizabeth M. Adler Executive Editor, The Journal of General PhysiologySharon Ahmad Executive Editor, Journal of Cell ScienceKurt H. Albertine Editor-in-Chief, The Anatomical RecordBruce Alberts Editor-in-Chief, ScienceDavid Botstein Founding Editor-in-Chief of Molecular Biology of the Cell; Director Lewis-Sigler Institute for Integrative Genomics, Princeton versityJulio E. Celis Editor-in-Chief, Molecular OncologyAna Maria Cuervo co-Editor-in-Chief of Aging Cell; Professor, Albert Einstein College of MedicineTracey DePellegrin Executive Editor, GENETICS and G3: Genes|Genomes|GeneticsDavid Drubin Editor-in-Chief, Molecular Biology of the Cell; Professor, University of California, BerkeleySir Alan Fersht, FRS Associate Editor, PNASBruce L. Goode Editor, Cytoskeleton; Professor, Biology Rosenstiel Basic Medical Sciences Research Center, Brandeis UniversitySharona Gordon Incoming Editor, Journal of General PhysiologyPeter Gunning President, Australian Society for Biochemistry and Molecular Biology; Editor-In-Chief, BioArchitecture, University of New South WalesRichard W. Hartel Editor-in-Chief, Journal of the American Oil Chemists' SocietyDennis W. Hess Editor, ECS Journal of Solid State Science & Technology and ECS Solid State LettersSteve Humphries Editor-in-Chief, Atherosclerosi, Official Journal of the European Atherosclerosis SocietyHowy Jacobs Chief Editor, EMBO ReportsMark Marsh Co-editor, Traffic; Director, Medical Research Council Laboratory for Molecular Cell BiologyMarc A. Marti-Renom Associate Editor at PLOS Computational Biology; National Center for Genomic Analysis and Centre for Genomic Regulation, Barcelona, SpainTom Misteli Editor-in-Chief, The Journal of Cell BiologyEric Murphy Editor-in-Chief, Lipids, a Journal of the American Oil Chemists' SocietyRichard N. Perham Editor-in-Chief, FEBS JournalAlberto Prestininzi Editor-in -Chief, Italian Journal of Engineering Geology and EnvironmentEdward N. Pugh, Jr. Editor, Journal of General PhysiologyBernd Pulverer Chief Editor, The EMBO Journal; Head of Scientific Publications, EMBOJordan Raff President, British Society of Cell Biology; Editor-in-Chief, Biology Open; Professor, Cancer Cell Biology, University of Oxford.Jean-Louis Salager Editor-in-Chief, Journal of Surfactants and DetergentsRandy Schekman Editor-in-Chief, eLifeTrina Schroer Co-editor, Traffic; Professor, Johns Hopkins UniversityJörg Schulz Editor-in-Chief, Journal of Neurochemistry; Chair and Full Professor, Department of Neurology, RWTH Aachen University, GermanyTom Stevens Co-editor, Traffic; Professor, University of OregonMarlowe Tessmer Senior Editor, The Journal of Experimental MedicineInder Verma Editor-in-Chief, Proceedings of The National Academy Of Sciences (PNAS)Michael Way Editor-in-Chief, Journal of Cell ScienceFelix Wieland Managing Editor, FEBS LettersMitsuhiro Yanagida Editor-in-Chief, Genes to Cells

Wellcome Trust

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 7

CBMR-SciComm

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

Editor comments about impacts factors

Bruce Alberts, Editor-in-Chief, Science, 17 May 2013“The impact factor, (a number calculated annually for each scientific journal based on

the average number of times its articles have been referenced in other articles), was never intended to be used to evaluate individual scientists, but rather as a measure of journal quality. However, it has been increasingly misused in this way, with scientists now being ranked by weighting each of their publications according to the impact factor of the journal in which it appeared. For this reason, I have seen CVs in which a scientist annotates each of his or her publications with its journal impact factor listed to three significant decimal places (for example, 11.345). And in some nations, publication in a journal with an impact factor below 5.0 is officially of zero value. As frequently pointed out by leading scientists, this impact factor mania makes no sense.”1

Tom Misteli, Editor-in-Chief, Journal of Cell Biology, 20 May 2013“The IF is pervasive in the scientific community. Scientists refer to it casually in

conversation to convince colleagues of the importance of their own papers, or they wonder how a paper ended up in “a journal with such a high Impact Factor.” Students and postdocs want to publish only in “high Impact Factor” journals, and the IF is frequently used in recruitment, tenure, and granting decisions when a candidate’s past publication performance is assessed…The IF was never meant to be used in that way!”2

1 Alberts, Science, 10.1126/science.12403192 Mistelli, JCB, 10.1083/jcb.201304162

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 8

CBMR-SciComm

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

DORA recommendations

• Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.

• Specific recommendations for funding agencies, institutions, publishers, and researchers:

• When involved in committees making decisions about funding, hiring, tenure, or promotion, make assessments based on scientific content rather than publication metrics.

• Wherever appropriate, cite primary literature in which observations are first reported rather than reviews in order to give credit where credit is due.

• Use a range of article metrics and indicators on personal/supporting statements, as evidence of the impact of individual published articles and other research outputs

• Challenge research assessment practices that rely inappropriately on Journal Impact Factors and promote and teach best practice that focuses on the value and influence of specific research outputs.

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 9

CBMR-SciComm

What is Impact?

CBMR-SciComm

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 10

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

Impact?

• Paper on sexually-sustained injury downloaded 40,000 times – impact?

• Randomized trial which proves that cheap antibiotic can prevent the spread of opportunistic infections – and ultimately, deaths – in children with HIV – impact?

• Review cited hundreds of times – impact?

• Arsenic life paper picked up and vigorously discussed among bloggers – impact?

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 11

CBMR-SciComm

What is Altmetrics?

CBMR-SciComm

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 12

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

Altmetrics

• Term coined by Jason Priem, UNC Chapel Hill, in 20101

• A form of article-level metrics (<- journal-level metrics)• Measuring things other than traditional citations

• “Creation and study of new metrics based on the Social Web for analyzing, and informing scholarship”2

• Track the impact of research output by looking at conversations, discussions, mentions, etc. generated based on a specific research paper

• Fast! Appears in days/week rather than years• Diverse! Scholarly communications across different products,

platforms and audiences

1 https://twitter.com/#/jasonpriem/status/25844968813

2 Priem et al, Altmetrics: A manifesto, (v.1.0), 26 October 2010. http://altmetrics.org/manifesto

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 13

CBMR-SciComm

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

Altmetrics - continued

• PLOS + another example• Nature + another example

• Altmetric.com – metrics provider• Cell• Bookmarklet for others

• Predictions – Twitter: Highly tweeted articles 11 times more likely to be highly cited than less-tweeted articles1

• ImpactStory

• PlumX

1 Eysenbach, J Med Internet Res 2011;13(4):e123

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 14

CBMR-SciComm

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

Why should I care / what can I do?

• Why do you publish? To ensure the next grant?

• “If you are a scientist, you care about about letting people read your science. If you don't, you're an alchemist!”

• Altmetrics will allow funding agencies and employers to making more informed decisions - rather than relying on gut feelings, antiquated systems

• Track your work!• Discover new “flavors” of impact beyond citation• Take part in discussions around your work

• Make a live CV!

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 15

CBMR-SciComm

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

More information

• Websites• altmetrics.org• StartPublicering.nu

• Papers• The Altmetrics Collections (PLOS One)• Riding the crest of the altmetrics wave• Impact factor: a valid measure of journal quality?• Altmetrics: Rethinking the Way We Measure

• Blogs• Martin Fenner, MD (PLOS)• Jason Priem

• Ask your colleagues in SciComm!

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 16

CBMR-SciComm

For at ændre ”Enhedens navn” og ”Sted og dato”:

Klik i menulinjen, vælg ”Indsæt” > ”Sidehoved / Sidefod”.Indføj ”Sted og dato” i feltet for dato og ”Enhedens navn” i Sidefod

CBMR Summer Retreat, Pharmakon, 23 May 2013Slide 17

CBMR-SciComm