Download - Building and Measuring Research Impact
Building and Measuring Research Impact
Luke GeorghiouVice-President for Research & Innovation
University of Manchester
Why are researchers asked to demonstrate impact?
• Principal motivations for assessing impact– Accountability for public expenditure– Justification against competing demands for resources– Learning and improving
• ‘The explicit assessment of impact in the REF for the first time demonstrates the importance placed on research outside the research community. It will enable universities to rigorously demonstrate the success of their research and the significant contribution it makes to the economy and society.'– David Sweeney, Director (Research, Innovation, and Skills) HEFCE
RCUK Mission for Societal and Economic Impact
• To advance knowledge, understanding and technology (including the promotion and support of the exploitation of research outcomes), and provide trained researchers
• To build partnerships to enhance take-up and impact, thereby contributing to the:– • economic competitiveness of the United Kingdom– • effectiveness of public services and policy– • enhancement of the quality of life and creative output of the nation.
• Mission and Statement of Expectation on Economic and Societal Impact– Research Council UK’s commitment to excellent research that extends
the boundaries of human knowledge remains as strong as ever. These documents signal a … recognises that publicly funded research should benefit us culturally, socially and financially.
Negative reactions
Petition decries 'impact' agenda in research11 June 2009
Scientists stage mock funeral outside parliament in funding protest
Science for the Future claims funding policies risk plunging British science and industry 'back into the Dark Ages'
Why resistance?
• Principal concern is that application of impact criteria will affect ex ante the direction of research– Funding decisions on grants– REF funding
• and that will result in work of lower quality, integrity and even impact– Argument that ‘blue-skies research’ ultimately creates
prosperity and well-being and that focus on immediate impact may be at expense of greater long-term impact
– Blue-skies is Anglicism translating as ‘investigator-driven, curiosity-driven or basic research’
Impact and Social Science and Humanities
• Worry in SSH communities that STEM subjects could be seen as more ‘impactful’ and secure larger share of resource
• Concerns grounded in the supposed predominance of a particular sequential model of impact– Discovery Development Commercialisation
– As opposed to interactive/co-production model more typical of SSH
– Also whether impact defined predominantly in economic terms
• In fact most social science disciplines founded in the expectation of research driving change in society
• Less clear in some parts of humanities but nonetheless strong commitment to culture
Response: British Academy Report
“Punching our weight: the humanities and social sciences in public policy making”
Taxonomies of Impact
• Two main ways to classify– Type of beneficiary– Nature of the impact
• Often mixed• Ongoing struggle with recognising public
engagement
REF Panel C Taxonomy• Impacts on creativity, culture and society
– Influencing knowledge, behaviours, practices, rights or duties • Economic, commercial, organisational impacts
– Businesses or other wealth creators• Impacts on the environment
– Natural, historic or built• Health and welfare
– Quality of life enhanced or harm mitigated• Impacts on practitioners and professional services
– Development and delivery of services and/or ethics• Impacts on public policy, law and services
– government, public sector and charity organisations and societies through implementation or non-implementation
Impact types by faculty at UoM
Expert
advic
e
Clinica
l Prac
tice
Consulting
Industry/N
on-Public Funds
Licen
sing
Spin-O
ut
Media covera
ge0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Engineering & Physical Sciences HumanitiesLife Sciences Medical & Human Sciences
Based on Research Performance Exercise Self-reporting
Impact in UoM vision and strategy
• Our work must have an impact beyond academia and yield economic, social and cultural benefits whenever the opportunity arises– Key relationships– Manchester heritage– Creating and demonstrating
impact– Commercialisation– Societal challenges and social
responsibility
Quality Impact
People
Focus Resources
Integrity Alignment
STRATEGY
Individual incentives at UoM
• Parity of esteem with curiosity-driven research for impact-generating, translation and knowledge transfer activities in promotion and PDR– applied research and development includes transfer of
intellectual property into the wider economy; translation of research findings into clinical solutions; development of innovation; research and consulting relationships with companies, government departments and other public bodies; and the enrichment of the wider culture ...
• Training & development – from doctoral training, through new academics
programme and beyond
Individual incentives continued
• Outside work policy – often seeding institutional links– register of interests to manage potential conflicts of
interests likely to arise• Radical IP policy– generous share to originators : 85% to originators up to
first £1m plus any re-invested into research, then 50% to originators
• Investment– Proof of principle and venture funds (eg UMIP Premier
Fund, social enterprise).
Rolling Research Profiling Exercise
Data base of all research-active staff (E Scholar repository) supporting annual reporting and
assessment of:• Research outputs (assessed by externally moderated
peer review NOT journal rankings or impact factors)• Research expenditure (benchmarked)• Research student supervision• Impact, engagement and esteem – Reported text boxes
Ability to deliver impact also rests in organisational competences
• Working with business and other non-academic partners requires:– Mutual trust and mutual benefit– Professional interface– Recognition and management of differences
• Works best in context of long-term strategic or ‘broadband’ relationship– Reduction of transaction costs
• Hidden but vital competence– Ability to configure multiple disciplines in seamless interdisciplinary
configurations to solve business and societal challenges– Key rationale for critical mass and economies of scope
Infrastructure is important
• Enterprise courses, conferences, staff and student business competitions and networking
• Developing regional innovation system and Corridor including cultural/creative zone
• Vectors – Policy@Manchester– Academic Health Sciences Networks
• Cultural institutiions
Living with REF Impact Assessment
• UoM submitted 186 REF Impact Cases • Some key challenges
– Identification • Especially of impacts where key players departed • No systematic organisational memory
– Comprehension• Getting academics to understand the detailed criteria
– Verification• Assembling credible supporting evidence
– Uncertainty• No track record on how criteria will be interpreted or how “reach
and significance” translate in to a scale across hugely different cases• Unclear boundaries – eg is it enough to demonstrate impact on a
policy or should the policy also be evaluated?
• A new stretch for modified peer review– But do the underlying assumptions of peer review hold?
Building Impact into your CV
• Consider engagement with user and stakeholder communities not as an add-on to research but as a potential source of ideas and insights even if you do not rely upon them for data
• Present your work in non-academic fora to interested parties eg at practitioner forums, via social and traditional media
• Profess the value of your field as well as your individual work• Ensure that interactions outside academic world are recorded
and note references in public documents• When engaging with non-academics consider whether it is
possible to collect evaluation evidence such as surveys or letters
• Monitor your presence in social media
Closing observations
• Not all research can be expected to have impact during the time it is being conducted and not all impact is realised by those performing the research
• Nonetheless where potential exists we have a duty to society to facilitate and realise impact from research we have already done and in some cases from research as it takes place
• The ability to generate and to record impact is a necessary organisational capability and a desirable individual attribute
• REF Impact assessment is a test and a hurdle but should not dominate what we do