demonstrating the impact of uk e-research; a research council perspective drs. astrid wissenburg,...
TRANSCRIPT
Demonstrating the impact of UK e-Research; a research council perspective
Drs. Astrid Wissenburg, ESRC
Workshop 11: Profiling UK e-Research: Mapping Communities and Measuring Impacts
UK e-ScienceALL HANDS MEETING 2008
Demonstrating the impact of UK e-Research; a research council perspective
Definitions
Context and drivers
Current approaches - examples
Outstanding issues
What is (economic) impact? (1)
“An action or activity has an economic impact when it
affects the welfare of consumers, the profits of firms
and/or the revenue of government. Economic impacts
range from those that are readily quantifiable, in terms of
greater wealth, cheaper prices and more revenue, to those
less easily quantifiable, such as effects on the
environment, public health and quality of life. “ (Treasury)
“improvements for UK society and the economy” (RCUK)
What is (economic) impact? (2)
“The concept of 'impact' in the social sciences applies to all
sectors: public, private and voluntary. It embraces economic
and societal impact in the sense of direct and often quantifiable
economic benefits; wider social impacts that will benefit society
more generally such as effects on the environment, public
health or quality of life; and impacts on government policy, the
third sector and professional practice. (….) Economic and
societal impact builds on academic impact: developing social
science knowledge and methods.” (ESRC)
Impact aspects
Academic and non-academic
On all sectors: public, private, voluntary – and
general public
Quantifiable and non-quantifiable
Direct and indirect
Policy, practice, products
Strategic drivers for impact agenda
Government’s Science and Innovation investment frameworks emphasize the maximisation of public investment in science on the economy
Research councils have been challenged to demonstrate an increase in their economic impact
Major research challenges are interdisciplinary, and require collaboration and co-production to ensure quality research with impact
Purposes of identifying and measuring impact To learn lessons, specifically to understand, improve
and support the processes which create impact
To make the case for the science budget
To inform funding decisions
Identifying and measuring impact
Growing portfolio of evaluation methodologies to ‘measure’ economic and societal impact:
Across all types of ‘research activities’ – including e-Research
At all levels: national science budget, activity type, funding mode, programmes, individual projects
Across the whole life-cycle of activities, including pre and post
Example 1: RCUK Economic Impact study (SQW/PA, 2007) Series of 18 case studies across training, research and
facilities investments from all research councils.
Methodology: stakeholder interviews was supplemented by
data and secondary research
Used a classification of impact types:
Development of human capital
Business and commercial
Policy
Quality of life
Example 1: RCUK study - impacts
Examples of impacts from case studies include:
Use of scientific facilities by industry:
direct income through selling access
indirect impact through businesses' exploitation of
knowledge that is developed by using the facilities.
Software development and licensing revenue
Spin-off companies
Example 1: RCUK study - findings
A wide diversity of impacts emerging from research
funding, many not part of the original rationale for the
specific investment
Not possible to robustly extrapolate from the case study
findings to total spend of research councils, partly because
the methodology is resource intensive and expensive
Lack of ‘management information’ about outputs from
investments, especially post project.
Example 2: DIUS study on large scale facilities (2008)Review of economic impacts relating to the location of large-scale science facilities in the UK
Five facilities covered
Analysis of economic impacts arising from employment, expenditure and knowledge transfer and scientific benefits
Focus on benefit of the location of facilities in the UK as opposed to access to similar facilities abroad.
Example 2: DIUS study on large scale facilities (2008)The major economic impacts arise from:
employment of relatively highly paid staff, most of whom reside close to the facility
the awarding of contracts to UK-based suppliers
Plus:
contributing to local technology clusters but generally a relatively small component
Some examples of transferring knowledge and technologies to suppliers, but no evidence that this is on a substantial scale
Example 3: ESRC policy and practice impact evaluationsCase study approach intended to cover different investments and different methodologies, including:
“Payback” method: this method examines the stages of a project throughout its life, and categorises its “paybacks”.
"tracking forward" from research outputs to the way in which these have been incorporated into practice in the public and private sectors.
an analysis of end of award reports to assess the potential for impact, followed by an analysis based on "unobtrusive" or "non-reactive" measures involving web-based searches, and databases of publications
a logic chain, mapping resource inputs to shorter term research outputs and to longer term outcomes.
Example 3: ESRC policy and practice impact evaluations qualitative studies with mixed methods captures the
complexities and allows for triangulation
models that anticipate the communication flows
between researchers and potential users provide a
framework for analysis, but linear models assume a
trail of evidence which is seldom present
Example 3: ESRC policy and practice impact evaluations – next steps
Economic impact study of 2 Research Centres:
Identify achieved economic impacts (through
research, dissemination, networking, etc);
Apply appropriate techniques to calculate values for
the economic impact of selected aspects
Present the results within a broader analysis of the
Centres’ policy and practice impacts;
Identify lessons for future economic evaluations
Methodological issues
How can we address the problems of
Attribution
Traceability
Measurability
Time lags?
How can we move beyond inputs and outputs as
proxies for actual impact?
How do we represent the receptiveness of users?
Research funders issues
Can/should we establish any robust baseline?
What is the relationship between individual projects to programmes to investment portfolios to the science budget?
What we can do:
More systematic data gathering of output data and ‘impact’ stories, during and post projects
More testing of economic impact valuations
Extract the lessons about good pre-conditions and practice for impact creation
E-Research Issues
Are the current approaches applicable to assess the
impact of e-Research?
Understand pathways to impact, both direct and
indirect, from e-Research
Impact of e-Research on the research process itself
and the implications for impact measurement
Can some e-Research tools be sued as part of a
portfolio to collect and analyse ‘impact’ information
