12/9/10 pilot assessment impact- paperwork findings of the expert panels- report + appendix lessons...
Post on 14-Dec-2015
212 Views
Preview:
TRANSCRIPT
12/9/10
Pilot assessment impact- paperwork
• Findings of the expert panels- report + appendix
• Lessons learned- feedback from pilot institutions
• Examples good case studies
• Original and revised guidance given to Institutions
• All on http://www.hefce.ac.uk/research/ref/impact/
12/9/10
Main outcomes• Case studies good way for assessing impact – across
disciplines and using same approach and weighting
• Case studies varied widely in nature and whether based around individuals or teams
• Main criteria used – ‘reach’ and ‘significance’- OK
• Impact statements were less valuable in discriminating good impact- needed to be more strategic
12/9/10
Earth Systems & Environmental Science
4* 3* 2* 1* U
UOA average 18 28 24 15 15
Institution A 0 50 0 0 50
Institution B 50 0 0 0 50
Institution C 25 0 25 50 0
Institution D 35 25 25 15 0
Institution E 25 25 50 0 0
Institution F 25 25 25 25 0
Institution G 0 0 25 25 50
Institution H 0 50 25 0 25
Institution I 0 35 30 0 35
Institution J 0 100 0 0 0
12/9/10
English Language & Literature
4* 3* 2* 1* U
UOA profile 19 30 30 19 2
Institution A 25 0 50 25 0
Institution B 0 0 20 80 0
Institution C 20 40 40 0 0
Institution D 0 0 100 0 0
Institution E 10 20 50 20 0
Institution F 0 20 80 0 0
Institution G 40 60 0 0 0
Institution H 25 50 25 0 0
Institution I 35 50 15 0 0
Institution J 40 0 60 0 0
Institution K 0 60 0 40 0
Institution L 20 30 20 15 15
Institution M 0 0 0 100 0
12/9/10
Issues identified with submissions
• Failure to demonstrate that the impact was based on high quality research (minimum 2*)
• Failure to provide transparent evidence- i)to link the impact with research, ii) demonstrate that impact had happened
• Many cases were ‘poorly narrated’
• Use of public engagement, special advisory work and policy as examples of impact – all acceptable but link with individual, group or departments OWN research must be clearly illustrated
12/9/10
Users• Panel chairs emphasised the vital role of users
on panel
• Most Case studies evaluated by 3-4 panel members (ratio 50:50) users to academics
• Most users will be recruited in 2013 to take part in assessment in 2014- brought in for the Impact assessment
• Some concern from HEIs re availability of sufficient high quality informed users
12/9/10
Issues with process -future changes
• Templates need to be altered to put description of research first
• Examples needed to be given- guidance too abstract
• Weighting not yet decided – consulting with community. Likely to be 15-20%.
• Impact statements may go into environment section
12/9/10
Lessons for us• Start now- time consuming and involves wide
range academics, admin and support staff
• Impact is based on historical activity – need good institutional memory
• Will probably be 1 case study per 10 FTEs
• Need academic champion(s) in each UoA
• Involvement central administrators essential
• High level academic committee to over see draft examples and development to final stage
12/9/10
Timescales• Main panels start to meet early 2011
• March 2011- broad framework for Impact Assessment published
• July 2011- guidance on submissions published
• Autumn 2011 - panels consulted on criteria
• January 2012- panel criteria and working methods published
top related