first the basic principles of bibliometric analysis

Measuring Quality and Impact of the Social Sciences

Concepts, Opportunities and Drawbacks

Pre-Conference of the 10th International Conference on Science and Technology Indicators

University of Vienna, September 17, 2008

Anthony F.J. van Raan Center for Science and Technology Studies (CWTS)

Leiden University

This presentation will highlight recent CWTS projects:

* Benchmarking & Evaluation* HEFCE * Identification of Excellence

From these recent studies we present empirical results for social science fields particularly concerning:

* WoS coverage * Characteristics of WoS publications* Characteristics of n-WoS publications* Bibliometric results and peer judgments

First the basic principles of bibliometric analysis

Basic Concept: Quality

Scientific performance relates to achieved quality in the contribution to the increase of our knowledge (‘scientific progress’)

(1) as perceived by others: peer review (2) as measured by advanced bibliometric analysis

Basic issues for research assessment, also in the social sciences:

* Objectivity* Transparency* How to handle interdisciplinarity, definition of fields* Different ways, prestige and intensity of publication* Role of co-authors in publications* Orientation of research: local vs. global* Language* Ageing of research results* PhD training* Time dimension of awards* Socio-economic impact

VOLUME 88, Number 13 PHYSICAL REVIEW LETTERS 1 April 2002

Truncation of Power Law Behavior in “Scale-Free”Network Modelsdue to Information Filtering

Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and LuísA. NunesAmaral11 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215

2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,UniversitàdiRoma “La Sapienza,”PiazzaleAldo Moro 2, I-00185, Roma, Italy

3 CEA-Service de Physique de la Matière Condensée, BP 12, 91680 Bruyères-le-Châtel, France(Received 18 October 2001; published 14 March 2002)

We formulate a general model for the growth of scale-free networks under filtering information conditions—that is, when the nodes can process information about only a subset of the existing nodes in the network. We find that the distribution of the number of incominglinks to a node follows a universal scaling form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size but also by a feature not previously considered, the subset of the network “accessible”to the node. We test our model with empirical data for the World Wide Web and find agreement.

DOI: 10.1103/PhysRevLett.88.138701 PACS numbers: 89.20.Hh, 84.35.+i, 89.75.Da, 89.75.Hc

There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on thedynamics of the system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the number of outgoing links at a given node have distributions thatdecay with power law tails [4–6]. It has been proposed [9] that the scale-free structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential attachment”[10] in which new nodes link to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a larger degree are more likely to become known.

































Weight?

networks leading, possibly, to different dynamics, e.g., for the initiation and spread of epidemics.In the context of network growth, the impossibility of knowing the degrees of all the nodes comprising the network due to the filtering process—and, hence, the inability to make the optimal, rational, choice—is not altogether unlike the “bounded rationality” concept of Simon [17].Remarkably, it appears that, for the description of WWW growth, the preferential attachment mechanism, originally proposed by Simon [10], must be modified along the lines of another concept also introduced by him—bounded rationality [17].

We thank R. Albert, P. Ball, A.-L. Barabási, M. Buchanan, J. Camacho, and R. Guimerà for stimulating discussions and helpful suggestions. We are especially grateful to R. Kumar for sharing his data. We thank NIH/NCRR (P41 RR13622) and NSF for support.

[1] S. H. Strogatz, Nature (London) 410, 268 (2001).[2] R. Albert and A.-L. Barabási, Rev. Mod. Phys. 74, 47 (2002).[3] S. N. Dorogovtsev and J. F. F. Mendes, Adv. Phys. (to be published).[4] R. Albert, H. Jeong, and A.-L. Barabási, Nature (London) 401, 130 (1999).[5] B. A. Huberman and L. A. Adamic, Nature (London) 401, 131 (1999); R. Kumar et al., in Proceedings of the 25th International Conference on Very Large Databases (Morgan Kaufmann Publishers, San Francisco, 1999), p. 639; A. Broder et al., Comput. Netw. 33, 309 (2000); P. L. Krapivsky, S. Redner, and F. Leyvraz, Phys. Rev. Lett. 85, 4629 (2000); S. N. Dorogovtsev, J. F. F. Mendes, and A. N. Samukhin, ibid. 85, 4633 (2000); A. Vazquez, Europhys. Lett. 54, 430 (2001).[6] M. Faloutsos, P. Faloutsos, and C. Faloutsos, Comput. Commun. Rev. 29, 251 (1999); G. Caldarelli, R. Marchetti, and L. Pietronero, Europhys. Lett. 52, 386 (2000); A. Medina, I. Matta, and J. Byers, Comput. Commun. Rev. 30, 18 (2000); R. Pastor-Satorras, A. Vazquez, and A. Vespignani, arXiv:cond-mat/0105161; L. A. Adamic et al., Phys. Rev. E 64, 046135 (2001).[7] F. B. Cohen, A Short Course on Computer Viruses (Wiley, New York, 1994); R. Pastor-Satorras and A. Vespignagni, Phys. Rev. Lett. 86, 3200 (2001); Phys. Rev. E 63, 066117 (2001).[8] F. Liljeros, C. R. Edling, L. A. Nunes Amaral, H. E. Stanley, and Y. Åberg, Nature (London) 411, 907 (2001).[9] A.-L. Barabási and R. Albert, Science 286, 509 (1999).[10] Y. Ijiri and H. A. Simon, Skew Distributions and the Sizes of Business Firms (North-Holland, Amsterdam, 1977).[11] G. Bianconi and A.-L. Barabasi, Europhys. Lett. 54, 436 (2001).[12] A. F. J. Van Raan, Scientometrics 47, 347 (2000).[13] We consider a modification to the network growth rule described earlier in the paper: at each time step t, the new node establishes m new links, where m is drawn from a power law distribution with exponent gout.[14] For n(I) = const, one recovers the scale-free model of Ref. [9].[15] It is known [11] that, for an exponential or fat-tailed distribution of fitness, the structure of the network becomes much more complex; in particular, the in-degree distribution is no longer a power law. Hence, we do not consider in this manuscript other shapes of the fitness distribution.[16] L. A. N. Amaral, A. Scala, M. Barthélémy, and H. E. Stanley, Proc. Natl. Acad. Sci. U.S.A. 97, 11 149 (2000).[17] H. A. Simon, Models of Bounded Rationality: Empirically Grounded Economic Reason (MIT Press, Cambridge, 1997).

Citing Publications

Cited Publications

All calculations are corrected for self-citations!

What do citations measure?

- Many studies showed positive correlations between citations and qualitative judgments

- In principle it is valid to interpret citations in terms of intellectual influence which is an important aspect of scientific quality

- Thus, the concepts of citation impact and scientific quality do not coincide ‘automatically’

WoS sub-universe8,000 j; 1,000,000p/y


Truncation of Power Law Behavior in “Scale-Free” Network Modelsdue to Information Filtering

Stefano Mossa,1,2 Marc Barthélémy,3 H. Eugene Stanley,1 and Luís A. Nunes Amaral11 Center for Polymer Studies and Department of Physics, Boston University, Boston, Massachusetts 02215

2 Dipartimento di Fisica, INFM UdR, and INFM Center for Statistical Mechanics and Complexity,Università di Roma “La Sapienza,” Piazzale Aldo Moro 2, I-00185, Roma, Italy


We formulate a general model for the growth of scale-free networks under filtering information conditions—that is, when the nodes can process information about only a subset of the existing nodes in the network. We find that the distribution of the number of incoming links to a node follows a universal scaling form, i.e., that it decays as a power law with an exponential truncation controlled not only by the system size but also by a feature not previously considered, the subset of the network “accessible” to the node. We test our model with empirical data for the World Wide Web and find agreement.


There is a great deal of current interest in understanding the structure and growth mechanisms of global networks [1–3], such as the World Wide Web (WWW) [4,5] and the Internet [6]. Network structure is critical in many contexts such as Internet attacks [2], spread of an Email virus [7], or dynamics of human epidemics [8]. In all these problems, the nodes with the largest number of links play an important role on the dynamics of the system. It is therefore important to know the global structure of the network as well as its precise distribution of the number of links.Recent empirical studies report that both the Internet and the WWW have scale-free properties; that is, the number of incoming links and the number of outgoing links at a given node have distributions that decay with power law tails [4–6]. It has been proposed [9] that the scale-free structure of the Internet and the WWW may be explained by a mechanism referred to as “preferential attachment” [10] in which new nodes link to existing nodes with a probability proportional to the number of existing links to these nodes. Here we focus on the stochastic character of the preferential attachment mechanism, which we understand in the following way: New nodes want to connect to the existing nodes with the largest number of links—i.e., with the largest degree—because of the advantages offered by being linked to a well-connected node. For a large network it is not plausible that a new node will know the degrees of all existing nodes, so a new node must make a decision on which node to connect with based on what information it has about the state of the network. The preferential attachment mechanism then comes into play as nodes with a larger degree are more likely to become known.

























Refs > nWoS

Compendex

Medline

non-WoS publ:

Books

Book chapters

Conf. proc.

Reports

ArXiv

Total publ universe

LNCS

































Source expansion

Target expansion

Scopus*CWTS is in license agreement negotiations with Scopus*CWTS currently compares Scopus- vs. WoS coverage*CWTS bibliometric algorithms can be applied

to Scopus data

Google Scholar

Network of publications (nodes)

linked by citations (edges)

Lower citation-density Higher citation-density

e.g., applied research, e.g., basic natural social sciences medical research

FCSmJCSm

CPP Expected values for normalization

Absolutely necessary but……are they appropriate?

Journal

Field = set of journals‘established fields’

scientific medium-grained structure

+ reference-based re-definition (expansion) of fields

CWTS applies two types of field definitions:

fields

ACTA GENETICAE MEDICAE ET GEMELLOLOGIAEAMERICAN JOURNAL OF HUMAN GENETICSAMERICAN JOURNAL OF MEDICAL GENETICSANIMAL BLOOD GROUPS AND BIOCHEMICAL GENETICSANNALES DE GENETIQUEANNALES DE GENETIQUE ET DE SELECTION ANIMALEANNALS OF HUMAN GENETICSATTI ASSOCIAZIONE GENETICA ITALIANABEHAVIOR GENETICSBIOCHEMICAL GENETICSCANADIAN JOURNAL OF GENETICS AND CYTOLOGYCANCER GENETICS AND CYTOGENETICSCARYOLOGIACHROMOSOMACLINICAL GENETICSCURRENT GENETICSCYTOGENETICS AND CELL GENETICSCYTOLOGIADEVELOPMENTAL GENETICSENVIRONMENTAL MUTAGENESISEVOLUTIONGENEGENETICAGENETICA POLONICAGENETICAL RESEARCHGENETICSGENETIKAHEREDITASHEREDITYHUMAN BIOLOGYHUMAN GENETICSHUMAN HEREDITYIMMUNOGENETICSINDIAN JOURNAL OF GENETICS AND PLANT BREEDINGJAPANESE JOURNAL OF GENETICSJAPANESE JOURNAL OF HUMAN GENETICSJOURNAL DE GENETIQUE HUMAINEJOURNAL OF HEREDITYJOURNAL OF IMMUNOGENETICSJOURNAL OF MEDICAL GENETICSJOURNAL OF MENTAL DEFICIENCY RESEARCHJOURNAL OF MOLECULAR EVOLUTIONMOLECULAR & GENERAL GENETICSMUTATION RESEARCHPLASMIDSILVAE GENETICATHEORETICAL AND APPLIED GENETICSTHEORETICAL POPULATION BIOLOGYEGYPTIAN JOURNAL OF GENETICS AND CYTOLOGYREVISTA BRASILEIRA DE GENETICAANNUAL REVIEW OF GENETICSJOURNAL OF CRANIOFACIAL GENETICS AND DEVELOPMENTAL BIOLOGYJOURNAL OF INHERITED METABOLIC DISEASEPRENATAL DIAGNOSISADVANCES IN GENETICS INCORPORATING MOLECULAR GENETIC MEDICINECHEMICAL MUTAGENS-PRINCIPLES AND METHODS FOR THEIR DETECTIONDNA-A JOURNAL OF MOLECULAR & CELLULAR BIOLOGYEVOLUTIONARY BIOLOGYTERATOGENESIS CARCINOGENESIS AND MUTAGENESISTSITOLOGIYA I GENETIKAADVANCES IN HUMAN GENETICSPROGRESS IN MEDICAL GENETICSGENETICS SELECTION EVOLUTIONMOLECULAR BIOLOGY AND EVOLUTIONSOMATIC CELL AND MOLECULAR GENETICSBIOTECHNOLOGY & GENETIC ENGINEERING REVIEWSEXPERIMENTAL AND CLINICAL IMMUNOGENETICSGENE ANALYSIS TECHNIQUESJOURNAL OF MOLECULAR AND APPLIED GENETICSJOURNAL OF NEUROGENETICSTRENDS IN GENETICSDISEASE MARKERSANIMAL GENETICSGENETIC EPIDEMIOLOGYJOURNAL OF GENETICS

Main field: Social and Behavioral Sciences

SOCI AL AND BEHAVI ORAL SCI ENCES

ECONOMICS AND BUSINESS AGRICULTURAL ECONOMICS & POLICY BUSINESS BUSINESS, FINANCE ECONOMICS INDUSTRIAL RELATIONS & LABOR

EDUCATIONAL SCIENCES EDUCATION & EDUCATIONAL RESEARCH EDUCATION, SCIENTIFIC DISCIPLINES EDUCATION, SPECIAL PSYCHOLOGY, EDUCATIONAL

MANAGEMENT AND PLANNING AREA STUDIES MANAGEMENT PLANNING & DEVELOPMENT

POLITICAL SCIENCE AND INTERNATIONAL RELATIONSPUBLIC ADMINISTRATION POLITICAL SCIENCE

PUBLIC ADMINISTRATION

PSYCHOLOGY PSYCHOLOGY, APPLIED PSYCHOLOGY, BIOLOGICAL PSYCHOLOGY, CLINICAL PSYCHOLOGY, DEVELOPMENTAL PSYCHOLOGY, EXPERIMENTAL PSYCHOLOGY, MATHEMATICAL PSYCHOLOGY, MULTIDISCIPLINARY PSYCHOLOGY, PSYCHOANALYSIS PSYCHOLOGY, SOCIAL

SOCIAL AND BEHAVIORAL SCIENCES, INTERDISCIPLINARY DEMOGRAPHY

SOCIAL ISSUES SOCIAL SCIENCES, BIOMEDICAL SOCIAL SCIENCES, INTERDISCIPLINARY

SOCIOLOGY AND ANTHROPOLOGY ANTHROPOLOGY ETHNIC STUDIES FAMILY STUDIES SOCIOLOGY WOMEN'S STUDIES

Major field, e.g. Economics & Business

journals

APPL PREV PSYCHOLAPPL PSYCHOL-INT REVBEHAV SCI LAWBRIT J GUID COUNSCAREER DEV QCOUNS PSYCHOLCYBERPSYCHOL BEHAVERGONOMICSEUR J PSYCHOL ASSESSEUR J WORK ORGAN PSYGROUP ORGAN MANAGEHUM FACTORSHUM PERFORMHUM RESOUR MANAGEINT J AVIAT PSYCHOLINT J OFFENDER THERINT J SELECT ASSESSJ APPL PSYCHOLJ APPL SPORT PSYCHOLJ BEHAV DECIS MAKING

All publication titles + abstracts (~30,000,000) have been grammatically parsed to enable bibliometric analysis by themes/concepts/ instruments and to create word-correlation based maps of science

clusterField = clusters of concept-related

publicationsnew, emerging often interdisc. fields

scientific fine-grained structure

Social Sciences Top-50 EU universities, their top-10% publications in this field

Now specific sub-field CPP/FCSm values can be calculated, for instance for research on democracy

But, obviously, the finer grained, the more ‘noisy’

Basic Performance Indicators

•P Ouput: Number of publications in internationally refereed CI-covered journals

•C Absolute Impact: Number of (self-ex) citations to these publications

•H Hirsch-index•CPP Output-normalized Impact: Average number of

cits/pub of the institute •JCSm Average number of cits/pub of the journal set

used by the institute•FCSm Average number of cits/pub of all journals of a

specific field in which the institute is active (FCSm)

•p0 Percentage of not-cited publications

CWTS Key Research Performance Indicators:

• JCSm/FCSm Relative impact of the used journal set

• CPP/JCSm Internat. journal-normalized impact• CPP/FCSm Internat. field & doc-normalized

impact

• Pt/Πt Contribution to the top-5, 10, 20,..%• P*CPP/FCSm Size & Impact Together: Brute Force

Applied research, engineering

Basic research

high FCSm

Up to factor ~20

high FCSm, but low JCSm

low FCSm

low FCSm, but high JCSm

High CPP

low CPP

Internal WoS-coverage of social science fields

results from HEFCE and Benchmark projects

Table 3.1: Internal coverage percentages of the Thomson Scientific/ ISI Citation Indexes

I nternal Coverage Percentage 80-100% 60-80% 40-60% <40% Biochem & Mol Biol Appl Phys & Chem Mathematics Other Soc Sci Biol Sci – Humans Biol Sci – Anim & Plants Economics Humanities & Arts Chemistry Psychol & Psychiat Engineering Clinical Medicine Geosciences Phys & Astron Soc Sci ~ Medicine

Internal WoS coverage of main fields of science

major field P 00-04 Avg Nr Refs Refs<1980 % Refs<1980 Refs Non-CI Refs CI % Refs CI

CLINICAL MEDICINE 5.376 31,18 8.138 5% 15.989 143.516 90%BIOL SCI: HUMANS 2.597 38,18 4.117 4% 7.484 87.546 92%BIOL SCI: ANIMALS & PLANTS 124 36,28 333 7% 659 3.507 84%MOLECULAR BIOLOGY & BIOCHEM 671 42,82 915 3% 1.559 26.255 94%PHYSICS AND ASTRONOMY 5 19,40 29 30% 15 53 78%CHEMISTRY 45 26,51 74 6% 143 976 87%MATHEMATICS 119 27,33 445 14% 1.019 1.788 64%GEOSCIENCES 33 32,42 66 6% 521 483 48%APPLIED PHYSICS AND CHEMISTRY 177 25,58 375 8% 864 3.289 79%ENGINEERING 313 25,97 727 9% 3.210 4.193 57%MULTIDISCIPLINARY 61 30,33 101 5% 150 1.599 91%ECONOMICS 471 36,56 1.961 11% 7.510 7.748 51%PSYCHOL, PSYCHIATRY & BEHAV SC 341 36,76 975 8% 2.983 8.578 74%SOCIAL SC RELATED TO MEDICINE 573 31,70 956 5% 4.836 12.374 72%OTHER SOCIAL SCIENCES 251 33,73 789 9% 4.185 3.491 45%HUMANITIES & ARTS 80 42,53 846 25% 1.889 667 26%

What is the internal WoS coverage and how is it calculated? Example: EUR 2000-2004

TABLE 1: SHARE OF UNL REFERENCES TO WEB OF SCIENCE PAPERS

Main Field P

00-06 Avg Nr

Refs %Refs <1980

Nr. Refs >1979

%Refs CI

CLINICAL MEDICINE 329 33.6 5% 10,461 87% BIOL SCI: HUMANS 495 35.2 7% 16,199 88% BIOL SCI: ANIMALS & PLANTS 318 36.9 12% 10,347 77% MOLECULAR BIOLOGY & BIOCHEM 656 39.2 7% 23,950 90% PHYSICS AND ASTRONOMY 281 25.0 17% 5,793 82% CHEMISTRY 796 37.8 12% 26,549 86% MATHEMATICS 115 18.8 16% 1,824 57% GEOSCIENCES 143 29.9 10% 3,842 61% APPLIED PHYSICS AND CHEMISTRY 723 18.8 11% 12,040 76% ENGINEERING 447 19.3 8% 7,951 45% MULTIDISCIPLINARY 19 27.2 9% 470 85% ECONOMICS 124 31.3 11% 3,468 55% PSYCHOLOGY, PSYCHIATRY & BEHAV SC 10 28.8 7% 267 45% SOCIAL SCIENCES RELATED TO MEDICINE 43 29.8 5% 1,215 74% OTHER SOCIAL SCIENCES 60 27.2 12% 1,439 47% HUMANITIES & ARTS 37 57.6 36% 1,365 17%

Internal WoS coverage (%) of submitted publications per UoA

SS&H Sports-related Subjects 698 65 SS&H Economics and Econometrics 1,688 51 SS&H Accounting and Finance 164 40 SS&H Geography 2,697 39 SS&H Business and Management Studies 3,205 36 SS&H Anthropology 299 35 SS&H Library and Information Management 366 35 SS&H Built Environment 560 34 SS&H Archaeology 327 31 SS&H Linguistics 199 30 SS&H Art and Design 194 28 SS&H Social Work 344 27 SS&H Sociology 933 24 SS&H Education 1,230 24 SS&H Social Policy and Administration 948 24 SS&H Town and Country Planning 520 23 SS&H Politics and International Studies 1,040 18

From: Moed, Visser, Buter, 2008

Table 3.3: Estimated CI-coverage (1991-2006), based on the extent to which references within CI -covered publications are also CI -covered Field 1991 1996 2001 2006 MEDI CAL & LIFE SCIENCES AGRICULTURE AND FOOD SCIENCE 66% 66% 73% 75% BASIC LIFE SCIENCES 87% 89% 93% 93% BASIC MEDICAL SCIENCES 76% 75% 80% 84% BIOLOGICAL SCIENCES 72% 74% 80% 82% BIOMEDICAL SCIENCES 86% 87% 90% 90% CLINICAL MEDICINE 82% 82% 85% 85% HEALTH SCIENCES 50% 47% 57% 62% NATURAL SCI ENCES ASTRONOMY AND ASTROPHYSICS 75% 79% 82% 86% CHEMISTRY AND CHEMICAL ENGINEERING 77% 80% 86% 88% COMPUTER SCIENCES 38% 37% 42% 43% EARTH SCIENCES AND TECHNOLOGY 60% 60% 69% 74% ENVIRONMENTAL SCIENCES AND TECHNOLOGY 46% 46% 55% 62% MATHEMATICS 58% 57% 58% 64% PHYSICS AND MATERIALS SCIENCE 75% 78% 81% 84% STATISTICAL SCIENCES 49% 46% 52% 58% ENGI NEERING SCIENCES CIVIL ENGINEERING AND CONSTRUCTION 37% 33% 34% 45% ELECTRICAL ENGINEERING AND TELECOMMUNICATION 54% 52% 52% 53% ENERGY SCIENCE AND TECHNOLOGY 54% 48% 53% 59% GENERAL AND INDUSTRIAL ENGINEERING 42% 37% 44% 54% INSTRUMENTS AND INSTRUMENTATION 67% 62% 71% 69% MECHANICAL ENGINEERING AND AEROSPACE 58% 53% 57% 64% LANGUAGE, INFORMATI ON AND COMMUNI CATI ON INFORMATION AND COMMUNICATION SCIENCES 32% 28% 29% 32% LANGUAGE AND LINGUISTICS 26% 33% 43% 40% SOCIAL AND BEHAVI ORAL SCI ENCES ECONOMICS AND BUSINESS 35% 36% 35% 43% EDUCATIONAL SCIENCES 27% 31% 30% 36% MANAGEMENT AND PLANNING 23% 24% 27% 36% POLITICAL SCIENCE AND PUBLIC ADMINISTRATION 17% 18% 20% 24% PSYCHOLOGY 59% 59% 66% 72% SOCIAL AND BEHAVIORAL SCIENCES, INTERDISCIPLINARY 33% 34% 36% 40% SOCIOLOGY AND ANTHROPOLOGY 22% 27% 29% 34% LAW, ARTS AND HUMANITIES CREATIVE ARTS, CULTURE AND MUSIC 17% 14% 16% 14% HISTORY, PHILOSOPHY AND RELIGION 24% 23% 25% 27% LAW AND CRIMINOLOGY 27% 32% 32% 31% LITERATURE 14% 12% 11% 11% MULTIDISCIPLINARY JOURNALS 78% 83% 87% 87%

Internal WoS coverage for all main fields of science

LANGUAGE, INFORMATION AND COMMUNICATION INFORMATION AND COMMUNIC SCIENCES 32 28 29 32 LANGUAGE AND LINGUISTICS 26 33 43 40 SOCIAL AND BEHAVIORAL SCIENCES ECONOMICS AND BUSINESS 35 36 35 43 EDUCATIONAL SCIENCES 27 31 30 36 MANAGEMENT AND PLANNING 23 24 27 36 POLITICAL SCIENCE AND PUBLIC ADMIN 17 18 20 24 PSYCHOLOGY 59 59 66 72 SOCIAL AND BEHAV SCIENCES, INTERDISC 33 34 36 40 SOCIOLOGY AND ANTHROPOLOGY 22 27 29 34

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1991

1996

2001

2006

1991

1996

2001

2006

1991

1996

2001

2006

1991

1996

2001

2006

1991

1996

2001

2006

1991

1996

2001

2006

1991

1996

2001

2006

AGRICULTUREAND FOODSCIENCE

BASIC LIFESCIENCES

BASIC MEDICALSCIENCES

BIOLOGICALSCIENCES

BIOMEDICALSCIENCES

CLINICALMEDICINE

HEALTHSCIENCES

References non-ISI

References ISI

purple: non-WoS reflight blue: CI ref

1991-2006

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

19

91

19

96

20

01

20

06

19

91

19

96

20

01

20

06

19

91

19

96

20

01

20

06

19

91

19

96

20

01

20

06

19

91

19

96

20

01

20

06

19

91

19

96

20

01

20

06

19

91

19

96

20

01

20

06

ECONOMICS ANDBUSINESS

EDUCATIONALSCIENCES

MANAGEMENTAND PLANNING

POLITICALSCIENCE AND

PUBLICADMINISTRATION

PSYCHOLOGY SOCIAL ANDBEHAVIORALSCIENCES,

INTERDISCIPLINARY

SOCIOLOGY ANDANTHROPOLOGY

References non-ISI

References ISI


1991-2006

External WoS-coverage of social science fields

results from HEFCE and Evaluation projects

TABLE 2: EXTERNAL COVERAGE FOR UPPSALA PAPERS 2002 - 2006

Research Unit Wos papersJournal Papers

Journal Coverage

Total papers (in journals,

procs, books)

Total paper

Coverage

UPPSALA 8.502 11.403 75% 16.436 52%

Arts 61 567 11% 1.089 6%Centre for Gender Research 17 37 46% 77 22%

Centre for Multiethnic Research 1 12 8% 43 2%Dep of ALM (Archives, Libraries, Museums) 3 20 15% 43 7%

Dep of Archeology and Ancient History 5 67 7% 142 4%Dep of Art History 1 12 8% 34 3%

Dep of Cultural Anthropology and Ethnology 6 94 6% 151 4%Dep of History 9 72 13% 152 6%

Dep of History of Science and Ideas 5 77 6% 120 4%Dep of Literature 3 140 2% 242 1%

Dep of Musicology 1 7 14% 18 6%Dep of Philosophy 10 29 34% 67 15%

The Uppsala Progr for Holocaust and Genocide Studies 1 4 25% 9 11%

Biology 1.265 1.383 91% 1.528 83%Dep of Bioorganic Chemistry 32 39 82% 39 82%

Dep of Cell and Molecular Biology 322 345 93% 366 88%Dep of Ecology and Evolution 390 419 93% 441 88%

Dep of Evolution, Genomics and Systematics 275 307 90% 353 78%Dep of Physiology and Developmental Biology 234 260 90% 320 73%

The Linnaeus Centre for Bioinformatics 44 45 98% 51 86%

Chemistry 911 951 96% 1.072 85%Dep of Biochemistry and Organic Chemistry 201 216 93% 223 90%

Dep of Materials Chemistry 316 328 96% 410 77%Dep of Photo Chemistry and Molecular Science 65 66 98% 69 94%

Dep of Physical and Analytical Chemistry 372 384 97% 422 88%

Earth Sciences 340 433 79% 711 48%

Educational Sciences 8 60 13% 174 5%

Engineering 622 679 92% 1.223 51%

Languages 23 412 6% 839 3%Dep of English 12 37 32% 70 17%

Dep of Linguistics and Philology 4 66 6% 200 2%Dep of Modern Languages 7 84 8% 173 4%

Dep of Scandinavian Languages 0 233 0% 412 0%

Mathematics and Computer Science 453 595 76% 1.161 39%Centre for Image Analysis 22 27 81% 124 18%

Dep of Information Technology 265 324 82% 783 34%Dep of Mathematics 167 248 67% 275 61%

Medicine 3.556 4.078 87% 4.396 81%Dep of Genetics and Pathology 535 564 95% 608 88%

Dep of Medical Biochemistry and Microbiology 509 537 95% 554 92%Dep of Medical Cell Biology 253 282 90% 293 86%

Dep of Medical Sciences 975 1.104 88% 1.193 82%Dep of Neuroscience 490 552 89% 608 81%

Dep of Oncology, Radiology and Clinical Immunology 524 562 93% 578 91%Dep of Public Heatlh and Caring Sciences 372 491 76% 532 70%

Dep of Surgical Sciences 443 556 80% 607 73%Dep of Women's and Children's Health 196 241 81% 250 78%

Pharmacy 763 815 94% 872 88%Dep of Medical Chemistry 168 178 94% 191 88%

Dep of Pharmaceutical Biosciences 357 371 96% 394 91%Dep of Pharmacy 282 312 90% 335 84%

Physics 1.057 1.194 89% 1.689 63%Dep of Astronomy and Space Physics 138 174 79% 283 49%

Social Sciences 380 1.083 35% 2.476 15%Dep of Business Studies 21 70 30% 316 7%

Dep of Domestic Sciences 25 59 42% 99 25%Dep of Economic History 0 52 0% 133 0%

Dep of Economics 41 93 44% 131 31%Dep of Education 6 22 27% 102 6%

Dep of Eurasian Studies 9 139 6% 211 4%Dep of Government 27 75 36% 180 15%

Dep of Information Science 31 48 65% 109 28%Dep of Law 5 164 3% 307 2%

Dep of Peace and Conflict Research 12 34 35% 156 8%Dep of Psychology 149 186 80% 281 53%

Dep of Social and Economic Geography 12 23 52% 58 21%Dep of Sociology 15 54 28% 170 9%

Inst for Housing and Urban Research 31 74 42% 242 13%

Wos papersJournal Papers

Journal Coverage

Total papers (in journals,

procs, books)

Total paper

Coverage

Social Sciences 380 1.083 35% 2.476 15%

Biology 1.265 1.383 91% 1.528 83%

What is the external WoS coverage and how is it calculated? Example: Uppsala 2002-2006

Candidates for Spinoza Award 2006, M€ 1.5

Field P total Books/ Total Ext WoSproceed WoS Coverage

Amerindian Languages 100 3 3%Medical Biochemistry 130 97 75%Computer Science 111 31 37 33%General Linguistics 108 22 30 28%Clinical Psychology 175 11 72 41%Theoretical High Energy Physics 178 3 131 74%Neuroscience 105 1 97 92%Entomology 134 110 82%Cell Biophysics/ Tumor I mmunology 225 193 86%Strategy & International Business 69 32 27 39%Biology 180 31 134 74%Zoology 115 13 91 79%Polymer Technology 151 17 121 80%Public International Law 108 6 11 10%Social Psychiatry 140 30 136 97%Experimental Physics 290 9 228 79%Molecular Genetics 52 5 38 73%Soft Condensed Matter 78 3 66 85%Economics 269 34 23 9%Bio-Chemisal Analysis Systems 102 11 90 88%Epidemiology 222 24 150 68%Psychiatry and Addiction 326 65 108 33%Child Neurology 121 10 107 88%Stochastics 63 7 44 70%Genetic Epidemiology 263 7 226 86%Environmental Biotechnology 200 4 188 94%Theoretical Physics 113 11 90 80%

From:

Van Leeuwen

2006

84% of the total number of publications submitted to the 2001 RAE from science-related departments were published in WoS-covered journals.

For Mathematics publications the WoS coverage is only slightly lower (82%),

It is substantially lower for Social Sciences and Humanities (25%)

From: Moed, Visser, Buter, 2008

WoS coverage of submitted publications per main field and per subject group

Field/subject group*

Total Nr.

Papers

% Papers

in WoS

Total Papers in J rnls

% Papers in J rnls

% WoS papers in set of J rnl

Papers

Per main field

Science 95,056 84.1 87,712 92.3 91.1

Mathematics 6,634 81.8 6,105 92.0 88.9

Soc Sci + Humanities 91,324 24.9 47,972 52.5 47.4

Per subject group*

Clinical Medicine* 16,541 96.6 16,412 99.2 97.3

Health Sciences* 10,621 85.0 10,078 94.9 89.6

Subjects allied to Health* 10,203 73.7 9,408 92.2 79.9

Biological Sciences* 16,694 93.5 16,252 97.4 96.0

Physical Sciences 17,190 88.0 16,462 95.8 91.9

Engineering & Computer Science 23,807 70.1 19,100 80.2 87.4

SS&H Econ and Econometrics 2,879 67.5 2,482 86.2 78.3

SS&H Geography 4,890 61.6 3,863 79.0 78.0

SS&H Sports-related Subjects 1,301 60.5 1,099 84.5 71.6

SS&H Town & Country Planning 1,478 38.1 1,083 73.3 52.0

SS&H Business & Management St 9,746 37.9 7,977 81.8 46.3

SS&H Library & Inform Manag 1,259 31.7 743 59.0 53.7

SS&H Sociology 3,519 29.2 1,846 52.5 55.6

SS&H Anthropology 1,180 27.5 523 44.3 62.1

SS&H Politics and Internat St 4,382 26.4 2,240 51.1 51.7

SS&H Social Policy & Administr 3,912 25.8 1,937 49.5 52.1

SS&H Built Environment 2,471 24.5 1,532 62.0 39.5

SS&H Social Work 1,642 22.9 887 54.0 42.4

SS&H Accounting and Finance 779 21.7 664 85.2 25.5

SS&H American Studies 475 19.8 189 39.8 49.7

SS&H Education 8,662 16.0 4,805 55.5 28.8

External WoS coverage (%) of submitted publications per UoA

y = 0.0087x

R2 = 0.9068

0%

20%

40%

60%

80%

100%

0 10 20 30 40 50 60 70 80 90 100

External WoS Coverage (%)

Inte

rna

l W

oS

Co

ve

rag

e

What is the correlation between internal and external WoS coverage?

Characteristics of WoS publications insocial science fields

results from HEFCE and Benchmark & Evaluation projects

PHYSICS

0

5000

10000

15000

20000

25000

30000

35000

Cits91 Cits92 Cits93 Cits94 Cits95 Cits96 Cits97 Cits98 Cits99 Cits00 Cits01 Cits02

Publications from 1991,….1995

time lag & citation window

* SOCIOLOGY

0

500

1000

1500

2000

2500


Main differences with the natural and medical sciences:

*Lower numbers (more than 1 order of magnitude….)

*Slower rise , broader peak and much slower decay (less hectics…)

BIBLIOMETRIC INDICATORS FOR MAJ OR FIELDS

University P Rank C Rank CPP/ Rank P*CPP/ Rank

FCSm FCSmCLINICAL MEDICINE

ERASMUS UNIV ROTTERDAM 5.376 23 77.806 21 1,56 53 8.413 22

LEIDEN UNIV 3.904 57 50.276 58 1,36 97 5.298 62RADBOUD UNIV NIJMEGEN 4.205 47 46.421 63 1,24 148 5.213 64UNIV AMSTERDAM 4.622 37 59.862 38 1,42 78 6.573 40UNIV GRONINGEN 3.021 96 32.463 105 1,23 156 3.718 102UNIV MAASTRICHT 3.120 90 39.274 82 1,39 85 4.343 81UNIV TILBURG 59 - 629 - 1,28 - 75 -UNIV UTRECHT 4.757 34 55.591 47 1,32 114 6.280 44VRIJE UNIV AMSTERDAM 3.682 63 47.934 61 1,46 74 5.360 59

ECONOMICS

ERASMUS UNIV ROTTERDAM 471 20 1.946 34 1,18 118 558 33

LEIDEN UNIV 37 246 227 190 1,52 74 56 201RADBOUD UNIV NIJMEGEN 78 187 276 179 0,92 174 72 187UNIV AMSTERDAM 330 41 1.647 45 1,47 83 486 48UNIV GRONINGEN 237 67 863 93 0,95 165 226 98UNIV MAASTRICHT 289 49 1.020 79 1,02 146 296 77UNIV TILBURG 430 25 1.684 42 1,18 117 509 43UNIV UTRECHT 104 160 283 177 0,80 210 83 177VRIJE UNIV AMSTERDAM 302 45 995 81 0,90 182 271 83

HIGHLY CITED PAPERS FOR MAJ OR FIELDS

University P01-03 Ptop20% Rank Ptop10% Rank Ptop5% Rank Ptop1% Rank

CLINICAL MEDICINE

ERASMUS UNIV ROTTERDAM 2.929 851 23 476 22 254 25 71 21

LEIDEN UNIV 2.152 574 62 313 60 150 72 27 90RADBOUD UNIV NIJMEGEN 2.318 598 58 309 62 164 63 24 103UNIV AMSTERDAM 2.611 745 35 398 36 212 39 44 48UNIV GRONINGEN 1.640 398 103 192 109 98 114 15 150UNIV MAASTRICHT 1.805 487 78 267 77 134 78 29 83UNIV TILBURG 29 10 - 3 - 2 - 0 -UNIV UTRECHT 2.642 738 37 382 42 217 38 32 72VRIJE UNIV AMSTERDAM 2.057 600 57 323 58 167 61 32 72

ECONOMICS

ERASMUS UNIV ROTTERDAM 282 68 34 29 49 9 75 1 83

LEIDEN UNIV 17 4 211 1 221 1 178 0 133RADBOUD UNIV NIJMEGEN 54 9 171 4 168 3 133 0 133UNIV AMSTERDAM 193 54 45 30 46 16 46 1 83UNIV GRONINGEN 147 24 97 9 114 6 96 1 83UNIV MAASTRICHT 166 26 90 17 80 6 96 1 83UNIV TILBURG 278 68 34 34 41 17 41 1 83UNIV UTRECHT 66 5 202 2 199 0 227 0 133VRIJE UNIV AMSTERDAM 186 34 71 15 89 8 83 0 133

FIGURE III.9:

IMPACT COMPARED TO WORLD SUBFIELD AVERAGE2000 - 2006

ECONOMICS

LSE

NORTHWESTERN

OXFORD

EUR

MANCHESTER

UvT

DUKE

CAMBRIDGEUvA

VU

UM

LEUVEN

VIRGINIA

RUG

PITTSBURGH

IMP COLL

MCGILLJOHNS HOPKINS

GENT

UTRECHT

RU

BARCELONA

TORINO

LAUSANNE

KINGS COLL

BASEL

GENEVE

MILANO

LEIDEN

NAPOLI

0,00

0,50

1,00

1,50

2,00

2,50

3,00

0 100 200 300 400 500 600

TOTAL PUBLICATIONS

Black coloured squares above (below) the horizontal reference line represent groups for which the impact (CPP) is significantly above (below) world average (FCSm)

CP

P/F

CS

m

P % C CPP/ FCSm

ECONOMICS

ECONOMICS 173,4 47% 507,0 0,90MANAGEMENT 86,7 24% 431,2 1,11BUSINESS 78,7 21% 384,2 1,41BUSINESS, FINAN 26,6 7% 80,8 1,23AGRIC ECON&POL 2,5 1% 6,5 0,96

EUR 200-2006 Benchmark Study

TABLE 6: HIGHLY CITED PAPERS FOR THE UNIVERSITY AND INSTITUTES

Research Unit P 0203 Ptop 5% E(Ptop 5%) A/E(Ptop 5%)

Biology 438 30 21,8 1,37Chemistry 344 35 17,2 2,03

Earth Sciences 104 9 5,3 1,69Engineering 186 20 9,3 2,15

Mathematics and Computer Science 181 16 8,9 1,80Medicine 1.432 95 71,8 1,32Pharmacy 308 17 15,5 1,10Physics 368 21 18,2 1,15

Social Sciences 151 10 7,5 1,33

Uppsala University 3.190 228 159,5 1,43

RESEARCH AND IMPACT PROFILE COMPARISON CHART 2000 - 2005

0,991700027

1,383060614

5,815597951

0,788964015

1,701706725

0,835714364

0,484413159

1,576815757

0,320689302

1,124854594

1,437596678

0,89885929

1,498792549

3,549813304

0,654775446

0,467699105

1,058451869

0,714722537

1,710102432

1,813498253

1,223234261

1,742351882

0,766388575

0,643706864

0,599794868

0,937883234

0,93637077

0,845662773

0,8652083

1,028331892

4,213728461

0,603420963

1,157731549

5,98887319

0,988357391

1,046249014

0,727394586

0,816961813

0,7734368

1,207450867

1,452590319

0,569773338

3,015952381

0,941907378

2,911953195

1,251511155

0,127077691

0,361855648

2,181555537

0,873929771

0,317647587

1,144437183

0,674202907

0,461296324

1,159763899

0,860002345

0,925778809

0,972944083

1,161701811

2,0% 1,5% 1,0% 0,5% 0,5% 1,0% 1,5% 2,0%

PSYCHIATRY

PSYCHOLOGY, MULT

* PSYCHOLOGY, SO

PUBL ENV OCC HLT

* PSYCHOLOGY, EX

* PSYCHOLOGY, CL

* PSYCHOLOGY, DE

* ANTHROPOLOGY

* EDUCATION *ED

# ASIAN STUDIES

* POLITICAL SC

# LANGUAGE&LING

* PSYCHOLOGY, ED

BEHAVIORAL SC

REHABILITATION

# HISTORY

* INFORM SC&LIBR

HEALTH CARE SC&S

* PUBLIC ADMIN

SPORT SCIENCES

* PSYCHOLOGY, AP

* LANGUAGE&LING

# RELIGION

* AREA STUDIES

# PHILOSOPHY

* SOC SCI,INTERD

* EDUCATION, SPE

MEDICINE, LEGAL

# ARCHAEOLOGY

* ECONOMICS

Percentage of Total Publication Output

IMPACT: LOW AVERAGE HIGH

LEIDEN UNIV UNIV ZURICH

Characteristics of non-WoS publications insocial science fields


Unit of Assessment Publication Type P p0(% ) C C/ P

Psychology WoS journal article 840 9 7155 8.5 Authored book 13 46 81 6.2 Chapter in book 44 70 54 1.2 Conf contribution 29 97 1 0.0 n-WoS journ article 93 85 68 0.7

Chemistry WoS journal article 1,281 6 16076 12.5 Authored book 2 50 9 4.5 Chapter in book 7 57 19 2.7 Conf contribution 1 100 0 0.0 n-Wos journ article 9 78 12 1.3

Physics WoS journal article 1,352 6 19947 14.8 Authored book 18 44 340 18.9 Chapter in book 12 75 17 1.4 Conf contribution 31 55 120 3.9 n-Wos journ article 8 50 9 1.1

Top-10% (of impact) of EU publications in Political Science, Economics, and Psychology

1997-2003, 4-y citation window (to calculate their impact)

From references all WoS-references removed, only non-WoS references (with freq > 2) have been analyzed

Total about 28,000

Percentage of references dating prior to 1980

0%

10%

20%

30%

40%

50%

60%

70%

Politicalscience

Economics Psychology

% <1980

% <1980 articles

% <1980 books

% <1980 handbooks

% <1980 theses

From: Nederhof, van Leeuwen, van der Wurff 2008

From these:

Books and journals dominant among post-1980

references to non-WoS items

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Political science Economics Psychology

% Books

% Journals

I tems with small frequencies among post-1980 references to

non-WoS items

% total items (> 1979)

0%

1%

2%

3%

4%

5%

6%

Political science Economics Psychology

% Handbook % Thesis % Proc % Reports % Working P

(> 1980)

Top-ranked publications according to document type

Field Books J.Articles Chapters Manuals Handbook Edit.vol Unknown

Political sc 84% 7% 0% 0% 0% 5% 5%

Economics 89% 3% 0% 0% 3% 0% 6%

Psychology 68% 3% 3% 24% 0% 0% 3%

Top-50 non-WoS >1980 references by document type

Bibliometric results and peer judgments


Qualitative versus quantitative assessment

peer review reputation may have strong influenceincludes 'tacit knowledge' (e.g., instrument building)includes credits: expectations, we believe that…, ahead of time…takes products other than journals papers into accountfashion and hypes perhaps less influential

bibliometric reputation much less influential analysis only 'codified knowledge'

no credits: only past performance, evidence-basedproducts other than journal papers less importantfashion and hypes perhaps more influential on the short term

TABLE 5: BIBLIOMETRIC STATISTICS FOR DEPARTMENTS AND INSTITUTES 2002 - 2006

CPP/ CPP/ JCSm/ SelfDepartment P C C+sc CPP Pnc JCSm FCSm FCSm Cit

Biology 1.265 8.174 10.814 6,46 28% 1,01 1,36 1,35 24%

Dep of Bioorganic Chemistry 32 82 170 2,56 34% 0,36 0,55 1,54 52%Dep of Cell and Molecular Biology 322 2.183 2.868 6,78 21% 0,92 1,22 1,32 24%

Dep of Ecology and Evolution 390 2.163 2.835 5,55 31% 1,00 1,42 1,41 24%Dep of Evolution, Genomics and Systematics 275 1.741 2.268 6,33 32% 1,05 1,36 1,29 23%Dep of Physiology and Developmental Biology 234 1.909 2.565 8,16 24% 1,12 1,50 1,33 26%

The Linnaeus Centre for Bioinformatics 44 271 372 6,16 36% 1,15 1,67 1,45 27%

Chemistry 911 4.603 6.311 5,05 35% 1,08 1,35 1,25 27%

Dep of Biochemistry and Organic Chemistry 201 951 1.229 4,73 30% 0,91 0,99 1,09 23%Dep of Materials Chemistry 316 1.228 1.815 3,89 42% 1,11 1,44 1,30 32%

Dep of Photo Chemistry and Molecular Science 65 292 432 4,49 35% 1,09 1,41 1,29 32%Dep of Physical and Analytical Chemistry 372 2.266 3.058 6,09 31% 1,15 1,53 1,34 26%

Earth Sciences 340 838 1.316 2,46 46% 1,07 0,94 0,88 36%

Engineering 622 1.847 2.763 2,97 49% 1,07 1,35 1,26 33%

Mathematics and Computer Science 453 843 1.240 1,86 55% 1,17 1,11 0,95 32%

Centre for Image Analysis 22 59 81 2,68 32% 0,84 0,71 0,85 27%Dep of Information Technology 265 589 868 2,22 54% 1,22 1,24 1,01 32%

Dep of Mathematics 167 195 291 1,17 59% 1,13 0,97 0,86 33%

Medicine 3.556 24.034 30.552 6,76 28% 1,08 1,22 1,13 21%

Dep of Genetics and Pathology 535 4.579 5.820 8,56 24% 1,04 1,37 1,31 21%Dep of Medical Biochemistry and Microbiology 509 3.623 4.946 7,12 23% 0,88 1,14 1,30 27%

Dep of Medical Cell Biology 253 1.004 1.467 3,97 34% 0,70 0,74 1,06 32%Dep of Medical Sciences 975 7.376 8.981 7,57 28% 1,20 1,38 1,15 18%

Dep of Neuroscience 490 2.872 3.872 5,86 27% 1,03 1,03 1,00 26%Dep of Oncology, Radiology and Clinical Immunology 524 3.684 4.771 7,03 26% 1,14 1,17 1,03 23%

Dep of Public Heatlh and Caring Sciences 372 1.568 2.030 4,22 35% 1,03 0,92 0,89 23%Dep of Surgical Sciences 443 3.272 3.856 7,39 29% 1,24 1,52 1,22 15%

Dep of Women s and Children s Healthｴｴ 196 579 763 2,95 38% 0,77 0,74 0,96 24%

Pharmacy 763 3.912 5.379 5,13 29% 1,05 1,11 1,06 27%

Dep of Medical Chemistry 168 1.155 1.535 6,88 24% 1,33 1,56 1,17 25%Dep of Pharmaceutical Biosciences 357 1.388 2.022 3,89 31% 0,85 0,81 0,96 31%

Dep of Pharmacy 282 1.560 2.078 5,53 30% 1,11 1,27 1,15 25%

Physics 1.057 4.082 6.279 3,86 39% 0,89 1,17 1,32 35%

Dep of Astronomy and Space Physics 138 688 1.047 4,99 26% 0,87 0,91 1,04 34%Dep of Neutron Research 68 63 232 0,93 62% 0,34 0,47 1,39 73%

Dep of Nuclear and Partical Physics 141 412 758 2,92 43% 0,74 0,85 1,15 46%Dep of Physics 633 1.894 3.011 2,99 42% 0,78 1,10 1,41 37%

Dep of Theoretical Physics 89 1.057 1.293 11,88 18% 1,52 2,51 1,65 18%

Social Sciences 380 1.549 1.948 4,08 43% 1,25 1,26 1,01 20%

Dep of Business Studies 21 44 52 2,10 62% 0,80 1,23 1,52 15%Dep of Domestic Sciences 25 117 156 4,68 24% 0,99 0,79 0,80 25%

Dep of Economics 41 75 88 1,83 49% 0,64 0,85 1,34 15%Dep of Government 27 54 73 2,00 52% 1,44 1,13 0,78 26%

Dep of Information Science 31 347 411 11,19 35% 3,61 2,40 0,66 16%Dep of Psychology 149 549 737 3,68 42% 0,96 0,99 1,03 26%

Inst for Housing and Urban Research 31 67 89 2,16 35% 0,92 1,16 1,26 25%

TABLE 5: BIBLIOMETRIC STATISTICS FOR DEPARTMENTS AND INSTITUTES 2002 - 2006

CPP/ CPP/ JCSm/ SelfDepartment P C C+sc CPP Pnc JCSm FCSm FCSm Cit

Biology 1.265 8.174 10.814 6,46 28% 1,01 1,36 1,35 24%

Dep of Bioorganic Chemistry 32 82 170 2,56 34% 0,36 0,55 1,54 52%Dep of Cell and Molecular Biology 322 2.183 2.868 6,78 21% 0,92 1,22 1,32 24%

Dep of Ecology and Evolution 390 2.163 2.835 5,55 31% 1,00 1,42 1,41 24%Dep of Evolution, Genomics and Systematics 275 1.741 2.268 6,33 32% 1,05 1,36 1,29 23%Dep of Physiology and Developmental Biology 234 1.909 2.565 8,16 24% 1,12 1,50 1,33 26%

The Linnaeus Centre for Bioinformatics 44 271 372 6,16 36% 1,15 1,67 1,45 27%

Chemistry 911 4.603 6.311 5,05 35% 1,08 1,35 1,25 27%

Dep of Biochemistry and Organic Chemistry 201 951 1.229 4,73 30% 0,91 0,99 1,09 23%Dep of Materials Chemistry 316 1.228 1.815 3,89 42% 1,11 1,44 1,30 32%

Dep of Photo Chemistry and Molecular Science 65 292 432 4,49 35% 1,09 1,41 1,29 32%Dep of Physical and Analytical Chemistry 372 2.266 3.058 6,09 31% 1,15 1,53 1,34 26%

Earth Sciences 340 838 1.316 2,46 46% 1,07 0,94 0,88 36%

Engineering 622 1.847 2.763 2,97 49% 1,07 1,35 1,26 33%

Mathematics and Computer Science 453 843 1.240 1,86 55% 1,17 1,11 0,95 32%

Centre for Image Analysis 22 59 81 2,68 32% 0,84 0,71 0,85 27%Dep of Information Technology 265 589 868 2,22 54% 1,22 1,24 1,01 32%

Dep of Mathematics 167 195 291 1,17 59% 1,13 0,97 0,86 33%

Medicine 3.556 24.034 30.552 6,76 28% 1,08 1,22 1,13 21%

Dep of Genetics and Pathology 535 4.579 5.820 8,56 24% 1,04 1,37 1,31 21%Dep of Medical Biochemistry and Microbiology 509 3.623 4.946 7,12 23% 0,88 1,14 1,30 27%

Dep of Medical Cell Biology 253 1.004 1.467 3,97 34% 0,70 0,74 1,06 32%Dep of Medical Sciences 975 7.376 8.981 7,57 28% 1,20 1,38 1,15 18%

Dep of Neuroscience 490 2.872 3.872 5,86 27% 1,03 1,03 1,00 26%Dep of Oncology, Radiology and Clinical Immunology 524 3.684 4.771 7,03 26% 1,14 1,17 1,03 23%

Dep of Public Heatlh and Caring Sciences 372 1.568 2.030 4,22 35% 1,03 0,92 0,89 23%Dep of Surgical Sciences 443 3.272 3.856 7,39 29% 1,24 1,52 1,22 15%

Dep of Women?s and Children?s Health 196 579 763 2,95 38% 0,77 0,74 0,96 24%

Pharmacy 763 3.912 5.379 5,13 29% 1,05 1,11 1,06 27%

Dep of Medical Chemistry 168 1.155 1.535 6,88 24% 1,33 1,56 1,17 25%Dep of Pharmaceutical Biosciences 357 1.388 2.022 3,89 31% 0,85 0,81 0,96 31%

Dep of Pharmacy 282 1.560 2.078 5,53 30% 1,11 1,27 1,15 25%

Physics 1.057 4.082 6.279 3,86 39% 0,89 1,17 1,32 35%

Dep of Astronomy and Space Physics 138 688 1.047 4,99 26% 0,87 0,91 1,04 34%Dep of Neutron Research 68 63 232 0,93 62% 0,34 0,47 1,39 73%

Dep of Nuclear and Partical Physics 141 412 758 2,92 43% 0,74 0,85 1,15 46%Dep of Physics 633 1.894 3.011 2,99 42% 0,78 1,10 1,41 37%

Dep of Theoretical Physics 89 1.057 1.293 11,88 18% 1,52 2,51 1,65 18%

Social Sciences 380 1.549 1.948 4,08 43% 1,25 1,26 1,01 20%

Dep of Business Studies 21 44 52 2,10 62% 0,80 1,23 1,52 15%Dep of Domestic Sciences 25 117 156 4,68 24% 0,99 0,79 0,80 25%

Dep of Economics 41 75 88 1,83 49% 0,64 0,85 1,34 15%Dep of Government 27 54 73 2,00 52% 1,44 1,13 0,78 26%

Dep of Information Science 31 347 411 11,19 35% 3,61 2,40 0,66 16%Dep of Psychology 149 549 737 3,68 42% 0,96 0,99 1,03 26%

Inst for Housing and Urban Research 31 67 89 2,16 35% 0,92 1,16 1,26 25%

Social Sciences 380 1.549 1.948 4,08 43% 1,25 1,26 1,01 20%

Dep of Business Studies 21 44 52 2,10 62% 0,80 1,23 1,52 15% 4,0Dep of Domestic Sciences 25 117 156 4,68 24% 0,99 0,79 0,80 25% 2,5Dep of Economics 41 75 88 1,83 49% 0,64 0,85 1,34 15% 4,0Dep of Government 27 54 73 2,00 52% 1,44 1,13 0,78 26% 3,5Dep of Information Science 31 347 411 11,19 35% 3,61 2,40 0,66 16% 4,5Dep of Psychology 149 549 737 3,68 42% 0,96 0,99 1,03 26% 3,5Inst for Housing and Urban Research 31 67 89 2,16 35% 0,92 1,16 1,26 25% 4,0

Social Sciences and Humanities

1 5 0.45 0.29 0.40 0.45 0.65

2 58 1.12 1.92 0.18 0.64 1.24

3a 291 0.92 1.36 0.17 0.68 1.20

3b 145 0.82 1.23 0.00 0.62 1.10

4 366 1.02 1.80 0.37 0.84 1.31

5 389 1.26 1.30 0.53 1.03 1.59

5* 151 1.34 1.27 0.64 1.09 1.68

Normalised citation impact parameters per subject group and per rating Normalised Citation Impact Distribution

RATING # Depts MEAN STD P25 MEDIAN P75

Comparison WoS vs. Scopus

results from one of the HEFCE projects: see tomorrow Martijn Visser and Henk Moed: “Comparing Web of Science and Scopus on a paper-by-paper basis”

Conclusion

Advanced bibliometric analysis is a powerful tool to make research assessment more objective, transparent and effective, particularly in the natural science and medical fields, and also in many of the engineering and social science fields but both internal and external WoS/Scopus coverage are absolutely necessary parameters to assess the validity of WoS/Scopus based measurements (including the non-WoS/Scopus publications…)

As always, never use it as a stand-alone tool. But also: it is an effective instrument for measuring interdisciplinarity, knowledge flows and knowledge diffusion -even for non-WoS/Scopus publications!

Thank you for your attention

more information: www.cwts.leidenuniv.nl

Appendix

According to an influential Swiss scientist:

Bibliometric investigations are clearly not very reliable…. In particular, the "frequency of citation" does not account for the quality of the researchers, because

(1) it depends more often on the social recognition of the researcher than excellence of his/her scientific work;

(2) it favors researchers who work on fashionable topics;

(3) it favors the fields of knowledge which traditionally publish shorter articles compared to those where publications are longer;

(4) it cannot differentiate between the fashion and the substance of a paper;

(5) it can favor the authors of "surveys", who are very frequently cited, compared to the authors of focused research papers;

(6) a position article or even an erroneous article can be criticized and consequently well cited.

Write your name on papers by your PhD students Ignore your publisher’s copyright: put your paper online Work in a popular area so that many others can cite you Write survey papers, not research papers Never change your established research area Avoid innovative and new (but risky) projects Chose catchy titles for your papers Emphasize quantity instead of quality Do not lose valuable time, avoid events like this one Concentrate on paper production, not good teaching Heavily cite you own (and your friend’s) papers Never publish more than a single ‘least publishable unit’ Cannibalize your old papers: refurbish and republish them

According to an influential Swiss scientist:

How to increase your ‘bibliometric values’

Main anecdotal objections against citation analysis

- Mendel Syndrome- Wittgenstein Syndrome- Lowry Effect- Einstein effect- Old boys clique- Disgusting anyway

A scientist has index h if h of his/her N papers have at least h citations each and the other (N-h) papers have no more than h citations each

Hirsch (h-) index AFJ van Raan =

18

Correlation of h-index (h) with number of citations (C)for all chemistry groups in the Netherlands

y = 0.394x0.4543

R2 = 0.8793

1

10

100

1 10 100 1000 10000

C

h

Correlation of h-index (h) with number of publications (P)for all chemistry groups in the Netherlands

y = 0.7293x0.5186

R2 = 0.4859

1

10

100

1 10 100 1000

P

h

Correlation of h-index (h) with CPP/FCSmfor all chemistry groups in the Netherlands

y = 6.9566x0.5331

R2 = 0.2161

1

10

100

0.10 1.00 10.00

CPP/FCSm

h

Large European University

(A&H)

APC

BIOL-AP

BIOL-HU

CHEM

CLM ECON

ENG

GEO

MATH

MOLB

(MULTI)

PHYS

PSY

SOC-MED

(SOC)

0

25

50

75

100

0255075100PUBLICATION RANK PTCL

CIT

AT

ION

I P

AC

T R

AN

K P

CT

L

Among top 25 % in publication output and citation impact

Top 25%

Bottom 25%

Impactranking

Publ.ranking Top 25%Bottom 25%

‘ Top’ research university

(SOC)

SOC-MEDPSY

PHYS (MULTI) MOLB

MATH

GEO

ENG ECON

CLM

CHEM

BIOL-HUBIOL-AP

APC

(A&H)

0

25

50

75

100

0255075100PUBLICATION RANK PTCL

CIT

AT

ION

I PA

CT

RA

NK

PC

TL

University has a top position

in each discipline

Bottom 25% Publ.ranking Top 25%

Top 25%

Impactranking

Bottom 25%

Citation-counting scheme based on‘roof-tile’ method:

Citation years 1995 1996 1997 1998 1999 2000 2001 20021995

1996

1997

1998

1999

2000

2001

2002

1995 1996 1997 1998

1996 1997 1998

1997 1998

1998

1996 1997 1998 1999

1997 1998 1999

1998 1999

1999

1997 1998 1999 2000

1998 1999 2000

1999 2000

2000

1998 1999 2000 2001

1999 2000 2001

2000 2001

2001

1999 2000 2001 2002

2000 2001 2002

2001 2002

2002

Example: The Lancet

Publs Cits

2000+01 2002 Art 784 7134Not 144 593Rev 29 232Subtot 957(a) 7959(b)

Let 4181 4264Edi 1313 905Other 1421 909Total 7872 14037 (c)

ISI IF

Citations in 2002__________ Citeable documents in 2000 and 2001

14037 (c) 957 (a)

IF=14.7

CWTS IF Citations to Art/Not/Rev in 2002 Art/Not/Rev in 2000 and 2001

7959 (b) 957 (a)

Citations to Art/Let/Not/Rev in 2002 Art/Let/Not/Rev in 2000 and 2001

7959+4264 957+4181

IF=8.3

IF=2.4

Manipulability of citation indicators proposed in this study

To which extent are our citation-based indicators sensitive to manipulation?

Can one increase actual citation impact by:

(1) Increasing author self citation?

Author self-citations are not included: increasing author self-citation has no effect

(2) Publishing in high impact journals?

A case study of 2,000 UK senior authors with >10 p/y revealed that journal impact explains ~20% of the variance in the citation impact rates. Journal impact is therefore not a dominant determinant of actual citation impact at the level of individual senior authors.

(3) Collaborate more intensively?

Some studies report positive correlation between number of authors and citation impact, but they ignore differences in authoring practices among research fields. Author self-citations are not included in this study. It all depends upon who collaborates with whom. There is also the issue of causality: ‘good’ research may attract high-impact collaborators.

(4) Publishing with US authors because they overcite their own papers?

Studies found no conclusive evidence that US scientists in science fields excessively cite papers originating from their own country.

(5) Publishing less, only the very best papers?

One would expect a higher citation impact per paper. Longer term effects of such a publication strategy are uncertain. PhD students need papers in their CV’s. It may become difficult for a group to attract good PhD students if its policy is to let them publish only a few papers. Another factor is that publications also enhance the visibility of a group’s research activities. If a group starts publishing substantially less papers, this may lead to a lower visibility and hence to a lower citation impact, even per paper.

(6) Making citation arrangements?

A high impact group receives its citations from dozens of different institutions. The distribution of citations amongst citing institutions is very skewed. The contribution of the tail of the distribution to the citation impact is relatively large. Making arrangements with a few institutions will not lead to a substantial increase in citation impact.

More information: www.cwts.leidenuniv.nl

mapping example:http://studies.cwts.nl/projects/leiden-benchmark

Application of Thomson-ISI Impact Factors for research performance evaluation is irresponsible

* Much too short ‘Citation window’* No Field-specific Normalization * No distinction between document types * Calculation errors/inconsistencies nominator/denominator* Underlying citation distribution is very skew:

IF-value heavily determined by a few very highly cited papers

* LANGUAGE&LING

0

100

200

300

400

500

600


* What is quality?* Numbers are order of magnitude lower > examples (e.g., profiles)* H-index example social sciences* National publications (also the case for engineering!)* Coverage* Figures about the role of books vs. journal papers (Uppsala data?)* Language* Citation window* non-WoS analysis, target known* non-WoS analysis, target unknown (CHE-2 results)* Societal relevance of social sciences > try sustainability maps

for social science themes!* Database problems (EC-ASSIST list)* Social science data from benchmark studies* Norwegian Association of Higher Education Institutions classification of sources* European Reference Index for the humanities journal classifications ERIH-ESF-HERA* Library Catalog Analysis: number of library copies per book title, e.g., Worldcat (Linmans); exploratory study of 43 catalogs in economics (ask HM…)* Use slides of HMs and TNs CHERPA presentations!* bibliometrics is more than an instrument of research performance analysis, it van also reveal patterns of knowledge development and diffusion

* Is the lifetime of a book longer than that of a journal article?* Flemish study on social sciences and humanities!* Nature of citations may be different* Hierarchy of books through reputation of publishers?* Results of our HEFCE analysis of the RAE 2001* Figures on p.37-40 in HEFCE Scoping report* Leiden Benchmarking: social sciences and humanities: order of magnitude,

ranking, trend, soc sc & hum profiles, ‘bolletjes’ charts *

Field N

Political science 4742

Economics 9062

Psychology 14132

TABLE 1: PUBLICATION TYPES FOR UPPSALA UNIVERSITY AND INSTITUTES 2002 - 2006

Conf proc Bk Ch Books Rep Pat Oth Coll (ed.) Proc (ed.)Research Unit Regular Non

CiteableCdyear 2007

Regular Rev Book Rev

Doct Lic

UPPSALA 8.502 270 4 2.845 56 474 2.773 2.260 381 291 102 826 49 344 15 42

Arts 61 16 - 492 14 168 106 416 65 34 2 36 - 34 3 -Biology 1.265 38 1 116 2 5 88 57 5 28 6 20 4 10 - 1

Chemistry 911 17 - 38 2 5 95 26 1 9 5 3 2 2 1 -Earth Sciences 340 27 2 92 1 - 226 52 1 6 4 26 - 17 - -

Educational Sciences 8 - - 52 - 8 57 57 11 13 - 22 - 12 - -Engineering 622 12 - 56 1 1 528 16 4 17 14 12 13 10 - 13Languages 23 21 - 383 6 67 142 285 30 17 - 18 - 52 - 2

Mathematics and Computer Sc 453 8 1 142 - 1 521 45 21 7 36 318 32 13 2 3Medicine 3.556 88 - 510 12 35 100 218 22 58 10 9 1 47 - -Pharmacy 763 6 - 52 - 1 21 36 5 9 3 1 4 6 - -Physics 1.057 11 - 127 10 3 472 23 15 8 13 35 - 7 - 21

Social Sciences 380 33 - 698 5 61 574 819 165 78 10 347 - 123 9 3Theology 5 2 - 169 3 123 10 230 40 11 1 4 - 16 - -

WoS Non- WoSJournal articlesJournal articles Thesis

Results of the HEFCE study of the bibliometric characteristics of the 2001 RAE from: Moed, Visser, and Buter, 2008 Science Mathem Soc+Hum Total -----------------+--------+--------+--------+ Authored book | 690 | 112 | 13657 | 14459 | 0.36 | 0.06 | 7.08 | 7.49 | 4.77 | 0.77 | 94.45 | | 0.73 | 1.69 | 14.95 | -----------------+--------+--------+--------+ Chapter in book | 1645 | 152 | 21515 | 23312 | 0.85 | 0.08 | 11.15 | 12.08 | 7.06 | 0.65 | 92.29 | | 1.73 | 2.29 | 23.56 | -----------------+--------+--------+--------+ Conference | 4186 | 180 | 2785 | 7151 contribution | 2.17 | 0.09 | 1.44 | 3.70 | 58.54 | 2.52 | 38.95 | | 4.40 | 2.71 | 3.05 | -----------------+--------+--------+--------+ Edited book | 218 | 9 | 3489 | 3716 | 0.11 | 0.00 | 1.81 | 1.93 | 5.87 | 0.24 | 93.89 | | 0.23 | 0.14 | 3.82 | n = publ type of a main field as % of all publs of all 3 main fields

(e.g., 13,657 / 193,014) n = publ type of a main field as % of same publ type of all 3 main

fields (e.g., 13,657 / 14,459) n = publ type of a main field as % of all publs of the main field (e.g., 13,657 / 91,324)

TREND ANALYSIS FOR ERASMUS UNIVERSITY

CLINICAL MEDICINE

Time Period P Rank C Rank CPP/FCSm RankP*

CPP/FCSm Rank

1997 - 2000 3.808 27 23.686 20 1,71 38 6.501,1 221998 - 2001 3.921 26 23.470 20 1,55 52 6.095,1 211999 - 2002 4.019 25 24.249 21 1,53 53 6.139,5 222000 - 2003 4.170 26 25.704 22 1,52 59 6.344,6 262001 - 2004 4.363 23 27.415 20 1,57 52 6.853,7 232002 - 2005 3.368 22 33.038 16 1,68 35 5.660,4 182003 - 2006 2.341 23 31.856 15 1,73 36 4.049,6 15

Time Period Ptop 20% Rank Ptop 10% Rank Ptop 5% Rank Ptop 1% Rank

1997 - 2000 1.115 22 588 23 335 20 79 211998 - 2001 1.095 23 561 27 301 26 73 251999 - 2002 1.090 24 584 25 318 24 76 252000 - 2003 1.150 22 632 22 344 24 92 20

ECONOMICS

Time Period P Rank C Rank CPP/FCSm RankP*

CPP/FCSm Rank

1997 - 2000 280 30 363 44 1,07 129 300,2 511998 - 2001 302 26 339 50 0,95 161 288,3 521999 - 2002 320 22 374 43 0,94 153 301,6 542000 - 2003 364 20 536 32 1,08 120 393,2 432001 - 2004 389 20 528 33 0,98 146 382,6 492002 - 2005 296 18 627 30 1,09 139 323,6 392003 - 2006 205 19 665 30 1,18 119 241,1 37

Time Period Ptop 20% Rank Ptop 10% Rank Ptop 5% Rank Ptop 1% Rank

1997 - 2000 62 50 23 68 12 65 3 541998 - 2001 65 45 25 67 11 75 1 911999 - 2002 72 41 28 60 12 73 1 912000 - 2003 90 33 39 46 15 59 2 73

Figure 1

Output and impact compared world field average1999 - 2006

Erasmus University and 20 benchmark institutions

NORTHWESTERN UNIV (USA)

DUKE UNIV (USA) UNIV N CAROLINA - CHAPEL HILL (USA)

DARTMOUTH COLL (USA) UNIV PENN (USA)

UNIV MARYLAND - COLLEGE PARK (USA)

MIT (USA) STANFORD UNIV (USA)

HARVARD UNIV (USA)

UNIV CHICAGO (USA)

STOCKHOLM SCH ECON (SWEDEN)

COPENHAGEN BUSINESS SCH (DENMARK)

LONDON BUSINESS SCH (GREAT BRITAIN)

UNIV OXFORD (GREAT BRITAIN)

INSEAD (FRANCE)

UNIV MAASTRICHT (NETHERLANDS)

UNIV GRONINGEN (NETHERLANDS)

UNIV TILBURG (NETHERLANDS)

UNIV AMSTERDAM (NETHERLANDS)

ERASMUS UNIV ROTTERDAM

(NETHERLANDS)

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

5.50

0 100 200 300 400 500 600 700

TOTAL PUBLICATIONS

Black coloured squares above (below) the horizontal reference line represent groups for which the impact (CPP) is significantly above (below) world average (FCSm)

CP

P/F

CS

m

Top-ranked publications according to

country of origin

Country of first author of Top 50 publ > 1980

Field US UK EU continent Other Unknown

Political sc 50% 20% 18% 5% 7%

Economics 71% 9% 9% 6% 6%

Psychology 50% 12% 21% 12% 6%

In the set of ‘best’ publications submitted to the 2001 RAE it was found that journal articles constitute 73% of submitted papers from all Subject Groups.

For science-related Units of Assessment we find 92%. The profile for Mathematics is quite similar to that for Science.

In Social Sciences and Humanities books are important publication sources. The shares of authored books and book chapters are 15 and 24%, respectively.

Most highly cited non-WoS items by document type

Book

Journal

Article

Hand-

book Manual

Soft-

ware

Proc-

eedings Thesis Other

Political sc 54 19 10 3 2 14 4 5

Economics 75 52 42 2 2 4 5 8

Psychology 150 112 30 430 32 11 11 2

The comparison of WoS and Scopus coverage of the 2001 RAE ‘best’ publications shows that Scopus coverage is especially better in the Subject Groups Subjects allied to Health (e.g., clinical dentistry, nursing, pharmacy), and to a lesser extent also in Engineering & Computer Science and Health Sciences.

In Clinical Medicine, Biological Sciences and Physical Sciences, however, Scopus coverage is slightly lower than WoS coverage.

from: Moed and Visser 2008, Appraisal of Citation Data Sources, HEFCE-report

Percentage of WoS papers found in Scopus

Publ Year WoS Database Segment

Nr WoS-covered J ournals

Nr WoS Articles and

Reviews

% WoS Articles and

Reviews found in Scopus

1996 Science 5,320 673,271 93

1996 Soc Sc & Hum 2,610 88,583 57

1996 Total 7,930 761,854 89

2005 Science 6,146 867,748 97

2005 Soc Sc & Hum 2,719 94,629 72

2005 Total 8,865 962,377 95

0

20

40

60

80

100

0 10 20 30 40 50 60 70 80 90 100

% WoS Papers found in Scopus

% W

oS

Jo

urn

als

1996

2005

Distribution of %WoS papers found in Scopus, science fields

WoS vs Scopus coverage of 2001 RAE ‘best’ publications Total

submitt publ

% in WoS

% in Scopus

Δ

Discipline

Science 95,056 84.1 84.4 0.3 Mathematics 6,634 81.8 80.1 -1.7 Social Sciences and Humanities 91,324 24.9 25.9 1.0

Total submitt

publ

% in WoS

% in Scopus

Δ

SSHU Town & Country Planning 1,478 38.1 57.4 +19.4 SSHU Social Work 1,642 22.9 36.7 +13.8 SSHU Accounting and Finance 779 21.7 34.9 +13.2 SSHU Built Environment 2,471 24.5 35.9 +11.4 SSHU Social Policy & Administr 3,912 25.8 34.1 +8.3 SSHU Politics and Internat St 4,382 26.4 34.6 +8.1 SSHU Sociology 3,519 29.2 37.0 +7.8 SSHU Business & Managem St 9,746 37.9 45.5 +7.6 SSHU Geography 4,890 61.6 68.6 +7.0 SSHU Education 8,662 16.0 22.5 +6.5 SSHU Econom & Econometrics 2,879 67.5 72.0 +4.5 SSHU Sports-related Subjects 1,301 60.5 63.4 +2.9 SSHU Library & Inform Manag 1,259 31.7 34.4 +2.7 SSHU Anthropology 1,180 27.5 28.0 +0.4

INTERNAL COVERAGE OF THE CITATION INDEX BY MAIN FIELD

Main FieldP

00-05Avg Nr

RefsRefs

<1980%Refs<1980

Refs Non-CI Refs CI %Refs CI

CLINICAL MEDICINE 3,893 33.3 6,950 5% 11,637 110,945 91%BIOL SCI: HUMANS 2,421 39.0 4,449 5% 6,447 83,588 93%BIOL SCI: ANIMALS & PLANTS 754 41.2 5,638 18% 6,611 18,805 74%MOLECULAR BIOLOGY & BIOCHEM 1,257 40.5 2,930 6% 3,968 44,001 92%PHYSICS AND ASTRONOMY 1,492 36.7 4,898 9% 7,555 42,320 85%CHEMISTRY 871 34.5 3,608 12% 3,717 22,693 86%MATHEMATICS 233 21.5 957 19% 1,680 2,375 59%GEOSCIENCES 134 40.4 578 11% 2,169 2,673 55%APPLIED PHYSICS AND CHEMISTRY 514 24.7 1,382 11% 2,081 9,256 82%ENGINEERING 373 21.5 686 9% 3,151 4,185 57%MULTIDISCIPLINARY 126 30.5 215 6% 339 3,291 91%ECONOMICS 35 38.9 160 12% 593 608 51%PSYCHOLOGY, PSYCHIATRY & BEHAV SC 633 40.3 2,789 11% 7,296 15,406 68%SOCIAL SCIENCES RELATED TO MEDICINE 292 28.9 597 7% 2,153 5,698 73%OTHER SOCIAL SCIENCES 291 34.9 1,469 14% 5,649 3,047 35%HUMANITIES & ARTS 220 38.7 2,477 29% 5,063 973 16%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

AGRICULTURE AND FOOD SCIENCE

BASIC LIFE SCIENCES

BASIC MEDICAL SCIENCES

BIOLOGICAL SCIENCES

BIOMEDICAL SCIENCES

CLINICAL MEDICINE

HEALTH SCIENCES

ASTRONOMY AND ASTROPHYSICS

CHEMISTRY AND CHEMICAL ENGINEERING

COMPUTER SCIENCES

EARTH SCIENCES AND TECHNOLOGY

ENVIRONMENTAL SCIENCES AND TECHNOLOGY

MATHEMATICS

PHYSICS AND MATERIALS SCIENCE

STATISTICAL SCIENCES

CIVIL ENGINEERING AND CONSTRUCTION

ELECTRICAL ENGINEERING AND TELECOMMUNICATION

ENERGY SCIENCE AND TECHNOLOGY

GENERAL AND INDUSTRIAL ENGINEERING

INSTRUMENTS AND INSTRUMENTATION

MECHANICAL ENGINEERING AND AEROSPACE

INFORMATION AND COMMUNICATION SCIENCES

LANGUAGE AND LINGUISTICS

ECONOMICS AND BUSINESS

EDUCATIONAL SCIENCES

MANAGEMENT AND PLANNING

POLITICAL SCIENCE AND PUBLIC ADMINISTRATION

PSYCHOLOGY

SOCIAL AND BEHAVIORAL SCIENCES, INTERDISCIPLINARY

SOCIOLOGY AND ANTHROPOLOGY

CREATIVE ARTS, CULTURE AND MUSIC

HISTORY, PHILOSOPHY AND RELIGION

LAW AND CRIMINOLOGY

LITERATURE

MULTIDISCIPLINARY JOURNALS

References ISI

References non-ISI


2006

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

AGRICULTURE AND FOOD SCIENCE

BASIC LIFE SCIENCES

BASIC MEDICAL SCIENCES

BIOLOGICAL SCIENCES

BIOMEDICAL SCIENCES

CLINICAL MEDICINE

HEALTH SCIENCES

ASTRONOMY AND ASTROPHYSICS

CHEMISTRY AND CHEMICAL ENGINEERING

COMPUTER SCIENCES

EARTH SCIENCES AND TECHNOLOGY

ENVIRONMENTAL SCIENCES AND TECHNOLOGY

MATHEMATICS

PHYSICS AND MATERIALS SCIENCE

STATISTICAL SCIENCES

CIVIL ENGINEERING AND CONSTRUCTION

ELECTRICAL ENGINEERING AND TELECOMMUNICATION

ENERGY SCIENCE AND TECHNOLOGY

GENERAL AND INDUSTRIAL ENGINEERING

INSTRUMENTS AND INSTRUMENTATION

MECHANICAL ENGINEERING AND AEROSPACE

INFORMATION AND COMMUNICATION SCIENCES

LANGUAGE AND LINGUISTICS

ECONOMICS AND BUSINESS

EDUCATIONAL SCIENCES

MANAGEMENT AND PLANNING

POLITICAL SCIENCE AND PUBLIC ADMINISTRATION

PSYCHOLOGY

SOCIAL AND BEHAVIORAL SCIENCES, INTERDISCIPLINARY

SOCIOLOGY AND ANTHROPOLOGY

CREATIVE ARTS, CULTURE AND MUSIC

HISTORY, PHILOSOPHY AND RELIGION

LAW AND CRIMINOLOGY

LITERATURE

MULTIDISCIPLINARY JOURNALS

References ISI

References non-ISI

purple: non-WoS reflight blue: CI ref2006

first the basic principles of bibliometric analysis

Documents