practical aspects of scientometrics september 2011€¦ · practical aspects of scientometrics:...

Practical aspects of ScientometricsSeptember 2011

Koenraad Debackere, K.U.Leuven

Practical aspects of Scientometrics:Reflections on “modern” science and innovation policy• Imperatives of modern science and innovation policy• Data and indicator base for science and innovation

policy• Deployment of the indicator base for science and

innovation policy• Final reflections

2

Imperatives of modern science and innovation policy

• In the wake of Joseph Schumpeter: Robert Solow and the econometric proof of technology (progress) as the “3rd” production factor the Solow-residual to explain economic growth beyond labor and capital (The Review of Economics and Statistics, Vol. 39, No. 3. (1957), pp. 312-320):

– Y=F(K,L) (i.e. Cobb-Douglas Y = αLβKγ)– Y=F(K,L).X(T) with X(T) a parameter measuring technical progress

(e.g. evolution in R&D expenditures)– R2: ±20% ==> ±80%

“Creative destruction” AND “Learning by doing and continuous innovation”… enabling firms to reap enormous benefits from institutionalizing and routinizing their innovation production function… also: William Baumol

3

Imperatives of modern science and innovation policy• The advent of the knowledge production function – Zvi Griliches, 1990

Addition to valuable

knowledge

Innovative performance

Innovative input

R&

D

expenditures

R&

D

personnel

u

Patents

Bibliom

etric indicators

New

products or processes

v1v2v3

Measure of grow

th

Measure of

productivity

Measure of

profitability

Mark

et a

nd

Co

mp

an

y C

hara

cteristics

w

4


• The ‘endogenous’ nature of science and technology development, OECD

5


• The advent of the Triple Helix --- Etzkowitz & Leydesdorff, 2000

6

Indicators for science and innovation policy

• Imperatives of modern science and innovation policy• Data and indicator base for science and innovation



7

Data and indicator base

• The standards:– Frascati Manual: “The internationally recognized methodology for collecting and using

R&D statistics, the Frascati Manual is an essential tool for statisticians worldwide. It includes definitions of basic concepts, data collection guidelines, and classifications for compiling statistics. This updated edition contains improved guidelines adjusted for changes in OECD economies, including measurement of service-sector R&D, R&D globalization, and R&D human resources.”

– Content: Chapter 1. Aim and Scope of the ManualChapter 2. Basic Definitions and ConventionsChapter 3. Institutional ClassificationChapter 4. Functional DistributionChapter 5. Measurement of R&D PersonnelChapter 6. Measurement of Expenditures Devoted to R&D (GERD, BERD, HERD, GOVERD)Chapter 7. Survey Methodology and ProceduresChapter 8. Government Budget Appropriations or Outlays for R&D

8


• The standards:– Oslo Manual: “The Oslo Manual is the foremost international source of guidelines for the

collection and use of data on innovation activities in industry. This third edition has been updated to take account of the progress made in understanding the innovation process, the experience gained from the previous round of innovation surveys, the extension of the field of investigation to other sectors of industry and the latest revisions of international standard classifications.”

– Content:Chapter 1. Objectives and Scope of the ManualChapter 2. Innovation Theory and Measurement NeedsChapter 3. Basic DefinitionsChapter 4. Institutional ClassificationsChapter 5. Linkages in the Innovation ProcessChapter 6. Measuring Innovation ActivitiesChapter 7. Objectives, Obstacles, and Outcomes of InnovationChapter 8. Survey Procedures

9


• The standards:– Canberra Manual: “The Manual on the Measurement of Human Resources devoted to

S&T (the "Canberra Manual") was issued in 1995. It was prepared in close co-operation between the OECD and the DGXII/Eurostat of the European Commission, other OECD Directorates, UNESCO and the International Labour Office (ILO), with the support of national experts. Drawing on best international and national practice and classifications, the "Canberra Manual" provides definitions of human resources devoted to science and technology in terms of qualification (levels and fields of study) and occupation and discusses a number of variables of policy interest.”

– No important modifications

10


• The standards:– PISA 2006: “Science Competencies for Tomorrow’s World presents the results from the

most recent Program for International Student Assessment (PISA) survey, which focused on science and also assessed mathematics and reading. … By showing that some countries succeed in providing both high quality education and equitable learning outcomes, PISA sets ambitious goals for others.”

– Content:Chapter 1. IntroductionChapter 2. A Profile of Student Performance in ScienceChapter 3. A Profile of Student Engagement in ScienceChapter 4. Quality and Equity in the Performance of Students and SchoolsChapter 5. School and System Characteristics and Student Performance in ScienceChapter 6. A Profile of Student Performance in Reading and Mathematics from PISA 2000 to PISA 2006

11


• In addition:– Bibliometric indicators: often WoS based because of presence of citation index --- challenge

to develop appropriate subject & institutional classification schemes for benchmarking input & output internationally, in valid & robust ways --- challenge to integrate new data sources (Proceedings, SCOPUS, …).

– Technometric indicators: based on the major patent systems EPO, USPTO, PCT --- once again a need for benchmark- & classification systems --- including textmining complex patent reference structures.

12


• Practical scientometrics --- the context:

13

Webometrics

Informetrics Mathematics/Physics

Library andInformation Science

Sociology of science

History of science

Economics

Services forResearch in

LibrarianshipResearch management

Scientometricsapplied

basic Life sciences





14

Deployment: capita selecta

• Subject classification for benchmarking (Glänzel & Schubert, 2003)

15

…


• Profiling scientific activity (Glänzel et al., 2008)

16


• The development of valid and robust normalized bibliometric indicator approaches for institutional benchmarking and profiling (Glänzel, Thijs, Schubert, Debackere, 2009)

17


• The development of valid and robust normalized bibliometric indicator approaches for institutional benchmarking and profiling (Glänzel, Thijs, Schubert, Debackere, 2009)

18

Deployment: capita selecta• Mapping and benchmarking patent performance (ECOOM, 2011)

19

Deployment: capita selecta• Benchmarking relative technological specialization --- Balassa Index applied to patents, countries

and Fraunhofer technology domains (ECOOM, 2011)

20


• The development of valid and robust econometric approaches to measure additionality of R&D subsidies (Aerts & Czarnitzki, 2008)

21


• The development of valid and robust econometric approaches to measure additionality of R&D subsidies (Aerts & Schmidt, 2008)

22


• Using combined bibliometric, patent and R&D data to explain technology transfer performance (Van Looy et al., 2011)

23





24

Final reflections

• Practical scientometrics can rely on a wealth of robust data sources with benchmark potential:

– WoS & Scopus & …– Patent data (USPTO, EPO, PCT, …)– CIS & R&D surveys– Regionalized data (Eurostat)

• Practical scientometrics should be clever regarding indicators and methods:– Relative and normalized publication and citation indicators, impact factors– Activity index– Relative specialisation index– Salton cosine measures– Robust subject classification schemes

25

Final reflections

• Complement these indicators and methods with:– Intelligent (limited) additional data collection for controls– Case studies on institutions & policies

• Practical scientometrics should also understand:– The amount of data “work” needed to do robust bibliometric analysis (data

cleaning, institutional cleaning, harmonizing data …)– The need for appropriate IT-infrastructures to support scientific and policy

oriented bibliometric analyses• The input, output & outcome debates and challenges

26

THANK YOU

Questions?

practical aspects of scientometrics september 2011€¦ · practical aspects of scientometrics:...

Documents