evaluation of the national natural science foundation of china
DESCRIPTION
Evaluation of the National Natural Science Foundation of China. Erik Arnold American Evaluation Association Anaheim, CA 3 November 2011. Some milestones. Cultural Revolution: S&T system effectively destroyed - PowerPoint PPT PresentationTRANSCRIPT
Evaluation of the National Natural Science Foundation of China
Erik ArnoldAmerican Evaluation Association
Anaheim, CA3 November 2011
2
Some milestones
• Cultural Revolution: S&T system effectively destroyed• 1978 National Science Conference: “modernisation of
S&T is the key of the four modernisations”; legitimacy but little money
• 1985-92 reforming the science system• CAS agency 1982-6; NSFC; marketisation of technology and
business; decentralisation of strategy to research performers; reform of HR system to introduce merit-based pay
• 1986, NSFC established, based on DFG/NSF model• 1992-8 integrating S&T to the economy, developing
industry, universities and institutes• 1998-2005 building a national innovation system
• Especially splitting off the industrially-focused parts of CAS, supporting SMEs, increased attention to human resource development and basic research
3
Medium- and Long Term Plan for National Science and Technology Development, 2006-2020• Goal: to make China an innovation-driven economy by
2020• High-priority clusters
• Technologies for water, energy and environmental protection• IT, advanced materials and manufacturing• Biotechnologies and their applications• Space and marine technology• Basic sciences and frontier technology - Raise basic research
to 15% of GERD by 2020• 16 mission-driven megaprojects
4
Eight thrusts
• A boost for investment in R&D• Tax incentives for investment in STI• Government procurement policy to promote innovation• Innovation based on assimilating imported advanced
technology• Capacity-building in generating and protecting IPRs,
standards• Building national infrastructure and platforms for STI• Cultivate and utilise talents for STI• Support endogenous innovation via financial measures
5
Key role of NSFC in basic research – the only bottom-up funder
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
02,0004,0006,0008,000
10,00012,00014,00016,00018,00020,000 National Major Science Engineer-
ing Construction Project
Knowledge Innovation Program
973 Program
985 Program
211 Program
Special Research Fund for the Doctoral Program
State Key Laboratory Program
NSFC Fund
Million RMB
6
NSFC main tasks
1. The combination of curiosity- and demand- driven research as a ‘dual driving force’. The first NSFC General Assembly stressed that its basic research funding was aimed to support economic development
2. Promoting the balanced, coordinated and sustainable development of academic disciplines in China
3. Emphasis on fostering talents 4. Facilitating international exchange and cooperation
in basic research
7
Evaluation objectives
1. Provide an independent assessment of the overall performance of NSFC’s funding and management during the past 25 years, with a truly global perspective
2. Present key findings, lessons learned and recommendations to improve the NSFC’s funding and management performance as well as to achieve excellence in management
3. Develop a set of forward-looking guiding ideas, based on an international perspective, supporting NSFC’s strategic role within the NIS of China
8
Method
• Review by an International Evaluation Committee• Academic chair• Thirteen researchers, including one evaluation professional
• Interviews with NSFC and stakeholders• Extensive background report prepared by the National
Centre for Science and Technology Evaluation (NCSTE)• Bibliometrics• Surveys• Interviews/focus groups• Document review
9
International Evaluation Committee• Prof Richard N ZARE (Chair),
Stanford University, Chemistry• Prof HAN Qide (Vice Chair), Vice
Chairman, National People’s Congress, Medicine
• Prof Ernst-Ludwig WINNACKER (Vice Chair), Human Frontier Science Programme , Biochemistry
• Prof Erik ARNOLD (Rapporteur), Technopolis Group; University of Twente, Research and Innovation Policy
• Prof LU Yonglong (Rapporteur), Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences, Environmental Science and Management
• Prof XUE Lan (Rapporteur), Tsinghua University, Science and Technology Policy and Management
• Prof Akito ARIMA, Chairperson, Japan Science Foundation, Nuclear Physics
• Dr Richard A ANTHES, University Corporation for Atmospheric Research, Atmospheric Science
• Prof Anthony K CHEETHAM, University of Cambridge, Materials Science
• Prof MA Zhiming, Academy of Mathematics and Systems Science, Chinese Academy of Science, Mathematics
• Prof Andrew (FA) SMITH, University of Adelaide, Agriculture, Food and Wine
• Prof Jeannette M WING, Carnegie Mellon University, Computer and Information Sciences
• Prof XU Zhihong, Peking University, Life Sciences
10
Stupendous growth in GERD (RMB billions)
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
0
100
200
300
400
500
600
Development
Applied Research
Basic Research
11
BERD = 73% of GERD, but there is a low proportion of basic research in GERD
Japan 07 Korea 08 USA 07 China 08 Canada 080%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%Other
Development
Applied
Basic
12
Building infrastructure, low labour cost component
China 08 Japan 08 Germany 08 Korea 08 France 080%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%Instruments, equipment
Land, buildings
Other current
Labour
13
Shifting the system towards research universities
14
A key innovation of NSFC was German-style peer review
15
Growth in projects and institutions funded across the research system
1986 1990 1995 2000 2005 20090
5,000
10,000
15,000
20,000
25,000Number of NSFC-funded institutions
Number of PI of NSFC-funded projects
16
NSFC moved from general to more focused instruments
1986 1991 1996 2001 2006 20090%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100% OtherInternational (Regional) Cooper-ation and Exchange fundNational Science Fund for Talent Training in Basic ScienceJoint Research Fund for Overseas Chinese (HK and Macao) ScholarsNational DYS fundScience Fund for Creative Research GroupScience fund for Less Developed RegionYoung Scientists FundJoint FundMajor Research PlanMajor ProgramKey ProgramGeneral Program
17
Age distribution of Principal Investigators
≦25 26-30
31-35
36-40
41-45
46-50
51-55
56-60
61-65
66-70
≧710
5
10
15
20
25
30
1986-1990
1996-2000
2006-2009
Age
%
18
Discipline development
Type of discipline
NSFC as Main Funding Sourcein 1980s/1990s (No. of Cases)
NSFC as Main Funding SourceToday
(No. of cases)Sole Source Main Source One of
Multiple Sources
Sole Source Main Source One of Multiple Sources
Traditional or strong disciplines
6 40 1 1 9 37
Emerging disciplines and interdisciplinary research
4 32 4 0 5 35
Weak disciplines 1 18 0 0 6 13
Total 11 90 5 1 20 85
19
Women and ethnic minorities
Year Female Applicants Minority Applicants
Number of applications
Share in total
Number of projects funded
Share in total Number of
applicationsShare in
total
Number of projects funded
Share in total
2008 11776 23.9% 1880 21.1% 1410 2.7% 312 3.5%
2009 14485 25.2% 2231 22.2% 2195 3.8% 363 3.6%
2010 17061 26.2% 3002 23.0% 2431 3.7% 470 3.6%
20
Staff workload
2001 2002 2003 2004 2005 2006 2007 2008 20090
100
200
300
400
500
600
0
5,000
10,000
15,000
20,000
25,000
30,000
35,000Average number of applications managed per person
Average number of funded projects managed per per-son
Average amount of funding man-aged per person
1,000 RMB
21
Administrative efficiency is very highCountry Organisation Year Administrative Cost as
% of BudgetCanada CIHR 2009 6%
SSHRC 2009 3.3%
China NSFC 2009 2%
Denmark Basic Research Council 2009 4.4%
Germany DFG 2009 2.5%
Sweden Swedish Science Council 2009 9%
UK EPSRC 2009 3.5%
MRC 2009 3.5%
USA NSF 2009 3%
22
Publications in the WoS19
8619
8719
8819
8919
9019
9119
9219
9319
9419
9519
9619
9719
9819
9920
0020
0120
0220
0320
0420
0520
0620
0720
0820
09
0
50,000
100,000
150,000
200,000
250,000
300,000
350,000 USARussiaJapanIndiaGermanyFranceChinaBrazilUK
23
Relative impacts of publications relative to the World
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
USA
Russia
Japan
India
Germany
France
China
Brazil
UK
World average
24
Copublications
Sweden
Netherlands
Russia
Italy
Taiwan
South Korea
Singapore
France
Australia
Canada
UK
Germany
Japan
USA
0 5000 10000 15000 20000 25000 30000 35000 40000
2004 to 20081999 to 2003
Source: Thomson-Reuters
25
Growth, copublications 1999/2003 to 2004/9
Italy
Russia
Germany
Japan
Taiwan
UK
France
Netherlands
USA
Sweden
Australia
Singapore
Canada
South Korea
0% 50% 100% 150% 200% 250% 300%
26
UK/China co-publications grow faster than publications
27
From the UK perspective, there’s a quality penalty to pay for cooperation (mean impact factors 2000-2005)
Source: RCUK
28
Surprisingly insular travel pattern
29
What the IEC said
• Increase the share of basic research in GERD• NSFC needs more staff and resources• Bigger grants, more calls for proposals• Strengthen panels: interdisciplinarity; involve more
foreigners• Ensure assessment is, and looks, confidential and
‘squeaky clean’• More international connections; International Advisory
Board• More flexible use of funding• More high-risk research
30
What the IEC didn’t talk about
• Outcomes and impacts of NSFC funding (!)• Discipline development• The rich tradition of consultation in programming• Linking bottom-up and top-down funding approaches
• Programme 1 versus Programme 2, in Swedish terminology• The systemic role of NSFC in developing the research
and innovation community• NSFC’s role in wider policy development and
implementation
31
Issues in the Chinese NIS
• Learning to use all the investments• Raising quality while growing the system• Doing novel research in a top-down culture• Squeezing out ‘influence’ – is this still an issue?• Increasing basic research• Science-industry links, absorptive capacity • Autarchy vs globalisation and learning from abroad
32
Thank you
technopolis |group| has offices in Amsterdam, Ankara, Brighton, Brussels, Frankfurt/Main, Paris, Stockholm, Tallinn and Vienna
http://www.technopolis-group.com/resources/downloads/reports/nsfc_evaluation_report.pdf