The Norwegian bibliometric model

Gjert Kristoffersen

2

Origin and structure

• Introduced by the Norwegian government in 2004 in order to make part of the funding of higher education institutions in Norway performance based

• Two main components– Production of ECTS credits

• Open system (the more credits produced, the more money)

– Research performance• Closed system – you need to improve compared to

other institutions in order to increase your revenue

3

Research performance

• Parameters– Number of PhDs – Amount of funding from Norwegian Research Council

and EU– Publication record as measured by the bibliometric

system• Underlying assumption

– The parameters that the performance is measured against reflect research quality

4

How much of the government funding is performance based?

• University of Bergen 2011– 1,8 % of the total grant

• Faculty of Humanities– 2,1 % of the total grant from the university– 13,8 % of the total grant when fixed costs such as

salaries, office rent etc. are deduced from the total– Since the faculty lost money on ECTS credit

production, the gain on the research part actually amounts to 19,4 %

5

The bibliometric system

• Administrated by the Norwegian Association of Higher Education Institutions

• Stated aim– “[A] system of documenting academic publishing that

will serve as the basis for the research component of the budgets for universities and university colleges. The purpose is to create a performance-based funding model for research and to encourage more research activity at universities and university colleges.”

6

Components

• A register of publication channels• A database and registration system for publications• Assignment of value points to different kind of

publications• Annual reports to the ministry • Effects on budgets two years later

7

Publication channels

• Two main types– Journals– Publishing houses

• Only national and international channels– Academic monographs (ISBN)– Academic articles in anthologies (ISBN)– Academic articles in journals (ISSN)

• Nominations by the national cooperation councils of each discipline renewed each year

• Peer evaluation a necessary condition for inclusion

8

Two levels

• Level 1– All publications that are defined as academic, non-

local and based on peer review– Humanities: 3850 journals

• Level 2– A subset of level 1 that is comprised by the leading

publishing houses and journals within a given discipline

– Humanities: 721 journals = 18,7 % • (Maximum = 20 %)

9

The database

• Cristin (http://www.cristin.no/no/index.html) • The registration of entries is partly automatic, e.g. from

Institute of Scientific Information (ISI)• But at least within the humanities, each author is obliged

to enter his or her own publications• Those entries where the publication channel is part of

the register, is automatically tagged as “point earning”• Also other kinds of “publications” can be entered, such

as newspaper articles, interviews on radio or TV etc, but these do not earn points

10

How much is each publication worth?

Publication type

Level 1 Level 2

Monograph 5 8

Article in anthology 0,7 1

Journal article 1 3

11

Results for my faculty

2005 2006 2007 2008 2009 2010

222 284 346 383 429 461

• The budget value depends on– Results of the other institutions, since there is only a fixed

amount to be shared– How your institutions passes on the results to the faculties– Net value 2009 on all three measures = about 900.000 euros =

almost 20 % of what is left when fixed costs have been covered

12

The model as a research quality evaluation metric

• One of three measures that will determine the budget result– Publications– Number of PhDs each year– Extent of external funding from national research council and EU

• Quality at two levels– Level 1: Peer review, national or international level– Level 2: The most prestigious channels as nominated by the

disciplines themselves• Limited scope: Only academic publications

– Textbooks excluded– Writing of peer reviews excluded– Dissemination to the general public excluded

13

Problem

• How should gains (and losses) be passed down in the organization?