12/9/10

Pilot assessment impact- paperwork

• Findings of the expert panels- report + appendix

• Lessons learned- feedback from pilot institutions

• Examples good case studies

• Original and revised guidance given to Institutions

• All on http://www.hefce.ac.uk/research/ref/impact/

12/9/10

Main outcomes• Case studies good way for assessing impact – across

disciplines and using same approach and weighting

• Case studies varied widely in nature and whether based around individuals or teams

• Main criteria used – ‘reach’ and ‘significance’- OK

• Impact statements were less valuable in discriminating good impact- needed to be more strategic

12/9/10

Earth Systems & Environmental Science

4* 3* 2* 1* U

UOA average 18 28 24 15 15

Institution A 0 50 0 0 50

Institution B 50 0 0 0 50

Institution C 25 0 25 50 0

Institution D 35 25 25 15 0

Institution E 25 25 50 0 0

Institution F 25 25 25 25 0

Institution G 0 0 25 25 50

Institution H 0 50 25 0 25

Institution I 0 35 30 0 35

Institution J 0 100 0 0 0

12/9/10

English Language & Literature

4* 3* 2* 1* U

UOA profile 19 30 30 19 2

Institution A 25 0 50 25 0

Institution B 0 0 20 80 0

Institution C 20 40 40 0 0

Institution D 0 0 100 0 0

Institution E 10 20 50 20 0

Institution F 0 20 80 0 0

Institution G 40 60 0 0 0

Institution H 25 50 25 0 0

Institution I 35 50 15 0 0

Institution J 40 0 60 0 0

Institution K 0 60 0 40 0

Institution L 20 30 20 15 15

Institution M 0 0 0 100 0

12/9/10

Issues identified with submissions

• Failure to demonstrate that the impact was based on high quality research (minimum 2*)

• Failure to provide transparent evidence- i)to link the impact with research, ii) demonstrate that impact had happened

• Many cases were ‘poorly narrated’

• Use of public engagement, special advisory work and policy as examples of impact – all acceptable but link with individual, group or departments OWN research must be clearly illustrated

12/9/10

Users• Panel chairs emphasised the vital role of users

on panel

• Most Case studies evaluated by 3-4 panel members (ratio 50:50) users to academics

• Most users will be recruited in 2013 to take part in assessment in 2014- brought in for the Impact assessment

• Some concern from HEIs re availability of sufficient high quality informed users

12/9/10

Issues with process -future changes

• Templates need to be altered to put description of research first

• Examples needed to be given- guidance too abstract

• Weighting not yet decided – consulting with community. Likely to be 15-20%.

• Impact statements may go into environment section

12/9/10

Lessons for us• Start now- time consuming and involves wide

range academics, admin and support staff

• Impact is based on historical activity – need good institutional memory

• Will probably be 1 case study per 10 FTEs

• Need academic champion(s) in each UoA

• Involvement central administrators essential

• High level academic committee to over see draft examples and development to final stage

12/9/10

Timescales• Main panels start to meet early 2011

• March 2011- broad framework for Impact Assessment published

• July 2011- guidance on submissions published

• Autumn 2011 - panels consulted on criteria

• January 2012- panel criteria and working methods published