Integration of Information Systems for the Benefit of Researchers and Policymakers

A Need for a European Approach and Coordination?

Erik ArnoldEuropean Parliament, Brussels26 March 2013

2

Why are we interested in information and measurement? • Drive from the New Public Management for performance

measurement• Increased effort to link research with innovation• A need for output data to feeding the growing use of

performance-based research funding systems• More use of evaluation in managing and measuring

funding programmes• Increased autonomy of research-performing institutions

leads to a desire for more management information at institutional level

• Open access to the results of publicly-funded research• ICT systems increasingly enable access to, and research

use of, others’ research materials

Stakeholders’ use of research information    Governments and

government agenciesResearch institutions

management & governance

    EU and National Governments

Ministries responsible for research/innovation

Government Agencies

Governing Bodies/ Councils

Executives/ Management

Research Groups / departments & institutes

Inform policies & strategies

 

 

 

Determine competitiveness / strategic positioning x x x x x x

Improve performance and quality x x x

Improve system functionality x x

Aid resource allocation x

Assess per-formance

Quality, sustainability, relevance and impact of research activity x x

Other

 

Investor confidence/value-for-money and efficiency x x x x x

Recruitment x x

3

Based upon: Expert Group on Assessment of University-Based Research, Assessing Europe’s University-Based Research, European Commission, DG Research, 2010

Types of system are proliferating

• Research information systems• Institutional level• Funder or national level

• Indicator systems and databases at national and European level

• National research evaluation databases• International bibliometric databases

• Web of Science, Scopus

• Web-based publication databases• Scholar Google – a game-changer?

• Open access repositories• Others, eg

• CV and patent databases4

Use of bibliometrics in performance-based funding

Method Unit of analysis Type of bibliometrics Country

Indicator-based Institution Papers Denmark

Finland

NorwayPapers & citations Belgium (Flanders)

Poland

Slovak republic

SwedenPeer review Department/ field of

researchUK RAE

ItalyInformed by metrics Poland

Czech RepublicIndividual Spain

New Zealand

5Based on Diana Hicks, 'Overview of models of performance-based research funding systems', Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings, OECD Working Party on Research Institutions and Human Resources (RIHR), 2010

Why would we need better systems?

• Better measurement of the broad impacts of research• Support better evaluation methods• More data availability• Better, more sophisticated indicators• Improved understanding of research and innovation

systems• Availability of fine-grained, internationally comparable

data• Obtain input data (especially funding), not just outputs• Enable policy coordination and measurement at a trans-

national level

6

Data sources for research and innovation assessments

7

Growing interlinkage of systems, enabled by emerging standards

8

Use and sources of tools for evaluation

9

Questions for today’s workshop

1. Is our view on the current landscape of systems for research-related information collection correct? Did we miss out something?

2. Is it desirable to interconnect the various types of information? Is it practical? What would be/are the benefits?

3. Is interconnection of national systems at European level desirable? Is it practical? What would be the benefits?

4. Is there a coordination deficit at European level?5. Where should coordination come from?

10

11

Thank you

technopolis |group| has offices in Amsterdam, Ankara, Brighton, Brussels, Frankfurt/Main, Paris, Stockholm, Tallinn and Vienna