five myths

Five Myths

About Funding Scientific Research (in Austria)

….and what Evaluation can do to make them more ‘Evidence Based’.

Five Myths

• „No Money for the Humanities and Social Sciences in Austria!“

• „There is a funding gap between basic research and applied research!“

• „Engeneering is treated unfair!“

• Networks, Fretworks

• Impacts – now!

Myth 1:

„No money in the humanities / social sciences“

• Humanites get a raw deal… (Der Standard, 31. Mai 2005)

• Social Sciences and Humanities are „starved out“ financially (Austrian Green Party, 13. Mai 2005)

• …marginalisation of the Humanities…“.(M. Nießen, DFG)

Mapping the Social Sciences & Humanities in Austria…

• No Evaluation / No Benchmarking Exercise in the field

• Lack of data– Contract research of the ministries?

• No vivid programme scene

• … but looking at further empirical evidence…

R&D in the higher education sector, 2002

Source: FTB 2005

136.0582.275,42.982203 >> Humanities

165.7552.718,43.775208 >> Social Sciences

301.8134.993,86.757411Social Sciences & Humanities

70.089847,51.06044Agriculture, Forestry & Veterinary Medicine

333.5166.025,67.284144Medicine (incl. clinics)

173.4932.690,63.502173Technical sciences

387.1934.865,26.469197Natural sciences

FTEheadcount

R&D expenditures in 1000 EUR

R&D personnel

number of R&D units 

Contract Research, 2003

Source: FTB 2005

  total sum in % bm:bwk bm:vit bm:wa

Natural sciences 11.099.561 19,6 8.794.489 617.055 187.097

Technical sciences 7.472.237 13,2 1.686.947 5.214.199 132.180

Medicine (incl. clinics) 13.264.064 23,5 12.848.845 89.969 -

Agriculture, Forestry & Veterinary Medicine 2.997.521 5,3 464.558 - 10.433

Social Sciences & Humanities 21.698.122 38,4 18.189.699 1.129.160 950.817

>> Social Sciences 14.735.356 26,1 11.226.933 1.129.160 950.817

>> Humanities 6.962.766 12,3 6.962.766 - -

FWF-project fundingAcceptance Rates, 1998-2003

Source: Streicher 2004

• Highest Acceptance Rate– Natural Sciences and Humanities: 58%

• Lowest Acceptance Rate– Agriculture and Social Sciences: 35%

• Funding Rates– Quite homogeneous– 70 % Human Medicine– 80% Humanities

A „Benchmarking Exercise“

Source: FWF Evaluation, Streicher, Schibany 2004

• Benchmark Project– Natural Sciences, male co-ordinator– Age 40-50, Size 150 – 250 k€– Approval Rate: 52,4%

Variable % Difference in approval rateTechnical Sciences - 8,5Human Medicine -15,1Agriculture, Forestry, VetMed -18,1Social Sciences -19,2Humanities + 4,5(A cautionary remark: It would be wrong to interpret the co-efficents causally)

A „Benchmarking Exercise“

Source: Streicher 2004

• Yes, projects of a different scientific flavour face significantly different chances– Against a benchmark project, Social

Sciences are rejected far more frequently– Humanities are (slightly but significantly)

more successful

Take into account…

• Classification• Structural Issues

– Age?– Fragmentation of Research Units?– Perspectives for younger researchers?– Researchers = working poor?

• Quality– Kind of Indicators

• ……

Heterogeneous average working loads (in % of total working hours)

Source: FTB 2005

 teaching &

training R&D other tasks

Natural sciences 29,5 64,4 6,1

Technical sciences 31,3 61,5 7,2

Medicine (incl. clinics) 16,8 36,7 46,5

>> without clinics 24,7 65,8 9,5

>> clinics 14,0 26,3 59,7

Agriculture, Forestry & Veterinary Medicine 25,6 57,0 17,4

Social Sciences & Humanities 45,0 47,4 7,6

>> Social Sciences 43,8 48,5 7,7

>> Humanities 46,5 46,1 7,4

Conclusions

• There is never enough money for doing research– No evidence, that Humanities / Social

Sciencies are treated unfair

• „Not enough money for the humanities / social Sciencies?“– This is an urban legend

Challanges for the future:

• Evaluators– Evaluators should be

• Sceptical, • suspicious of everybody

– Triangulation is necessary!– Quantitative Methods are valuable sources of

information

• Stakeholders– Ask the big questions (from time to time), too.– Give the evaluators the degrees of freedom to answer

these questions.

Mythos 2:

Funding Gaps between basic and applied sciences

„Funding Gap“

Basic Sciences Applied Sciences

Wissenschaftin A

Sciencein A

Wirtschaftin A

Economyin A

Dream Nightmare Reality

Risk Aversity & FFF

• Overall [FFF] tends to take too little risk.

• FFF funding practice is risk-averse.

• [The linear model] is a misleading oversimplification that encourages us to make poor policy decisions.

Source: FFF Evaluation, Arnold 2004

Source: FFF Evaluation, Jörg 2004

Source: FFF Evaluation, Arnold 2004

FWF Projects: Commercial Output & Usability

41%: results are relevant for industry

30%: important lab results

20%: working prototypes exist

13% research results are suitable for commercialization straight away

Source: FWF Evaluation, Streicher, Schibany 2004

Basic Sciences Applied Sciences

BRIDGE: Translational Research & Brückenschlagprogramm

Verkehrstechnologien: ISB & A3

Weltraum: ASAP & ARTIST

Luftfahrt: TAKE OFF

Informationstechnologien: FIT-IT

Nanotechnologie: Nano-Initiative

Kplus

K-Ind / Knet

CDG

18.9

K-Ind ?

7.3 (2003)

8

11.6

3.5

5.9

10.8

5.11 (2004- Translational FWF)

101.51 127.15

~ 70 Millionen €

Conclusions

Source: mid term Evaluation FIT-IT, 2005

• There is no funding gap (anymore)

• „Funding gap: No guiding principle for policymakers (anymore)

• Funding Gap: Urban Legend II

Challenges for the Future I

• In a NIS, there is the need for Evaluation of Systems (from time to time)

• In a NIS, there is the need for Evaluation of Portfolios (from time to time)

Challenges for the future II

• Room for „curiosity driven Evaluation“• Methodological Development

– Evaluation is no pure science, but– It is no consulting business, too.– Of cause, Evaluation must have a sound scientific basis

• Ensure degrees of freedom– Budget!– TORs

• Fight Evaluation Fatigue– Realistic expectations– sufficent time spans

Next Steps

Paper, part of the conference….

„New Frontiers in Evaluation“

www.fteval.at/conference06

24./ 25. April 2006

Vienna, Austria

Team

Michael Dinges, Joanneum Research

Michaela Glanz, WWTF

Brigitte Tempelmaier, WWTF