Research Excellence Framework 2014: A briefing for

Staff

Professor Ros FoskettDeputy Vice Chancellor

History of Research Assessment Undertaken periodically on behalf of the Funding Councils Determines the QR (Quality-rated) funding for each HEI Previous exercises in 1986, 1989, 1992, 1996, 2001, 2008 Set up under Thatcher during tight budgetary restrictions. Originally called the Research Selectivity Exercise, the

Research Assessment Exercise (RAE) and now the Research Excellence Framework (REF)

Submission for REF 2014 will cover the period January 1st 2008 – December 31st 2013.

What is it? Process of expert review by academic peers of an

institution’s research Assessment by four Main Panels covering the spread of

academic disciplines. Panel A: Medicine, Health and Biological

Sciences Panel B: Physical Sciences, Mathematics,

Computing, Engineering Panel C: Social Sciences Panel D: Arts and Humanities

How is the REF used?By Government

To inform research funding allocations by the four UK HE funding bodies (approximately £2 billion per year)

Provide accountability for public funding of research and demonstrate its benefits

By Universities

To demonstrate that funding and reputation are contestable

To help frame their strategic direction

External environment

To provide benchmarks and reputational yardsticks

To provide comprehensible ratings of research excellence in all disciplines in HEIs across the UK

Key changes since the 2008 RAE Inclusion of assessment of impact

Fewer Units of Assessment (UoAs)/panels, operating more consistently

Strengthened equality and diversity measures

Revised eligibility criteria for staff

Addition of (limited) use of citation data in some UoAs

Removal of ‘esteem’ as a distinct element

Revised approach to ‘environment’ and data collection

Increased ‘user’ input; and an integrated role for additional assessors

Publication of overall quality profiles in 1% steps

Timetable

2011• Panels appointed

(Feb)

• Guidance on submissions (Jul)

• Draft panel criteria for consultation (Jul)

• Close of consultation (Oct)

2012• Final panel criteria

and methods (Jan)

• HEIs submit codes of practice (Jul)

• Requests for multiple submissions (Dec)

• Survey of submission intentions (Dec)

2013• Launch REF

submissions system (Jan)

• Recruit additional assessors

• Staff census date (31 Oct)

• Submissions deadline (29 Nov)

2014• Panels assess

submissions

• Publish outcomes (Dec)

The assessment framework

Overall quality

Outputs

Maximum of 4 outputs per researcher

Impact

Impact template and case studies

Environment

Environment data and template

65% 20% 15%

Assessment

Criteria for assessing: quality of outputs

originality, significance and rigour impacts

reach and significance Environment

vitality and sustainability

AssessmentOutputs Impact Environment

4* World-leading Outstanding impacts Conducive to producing research of world-leading quality

3* Internationally excellent Very considerable impacts

Conducive to research of internationally excellent quality

2* Internationally recognised

Considerable impacts Conducive to research of internationally recognised quality

1* Nationally recognised Recognised but modest impacts

Conducive to research of nationally recognised quality

U/C Falls below quality of ‘nationally recognised’ or does not meet the definition of ‘research’

Impact of little or no reach and significance or ineligible according to the definition

Not conducive to producing research of national quality

Objectives for UW REF2014 To demonstrate an increase in research activity in terms of breadth

and depth for reputational enhancement

To maximise the submission as far as possible in terms of QR funding received (which will only be for 3* and 4* research in REF)

To submit a greater number of members staff and a higher proportion

of the academic staff than in RAE 2008

To make a submission to an increased number of Units of Assessment than in RAE 2008

UW REF2014We will aim to: Be as inclusive as possible yet aspire to maximise reputation and

income

Maximise the fundable research 3* and 4* outputs and impact case studies

Include 1* and 2* research for reputational reasons

Exclude anything which might attract an Unclassified score

RAE 2008 ResultsUoA Staff FTE

returned% of research activity in the submission judged to meet the standard for:

4* 3* 2* 1* U/C

Allied Health Professions and Studies

8.2 0 5.0 40.0 45.0 10.0

English Language and Literature

5.4 5.0 20.0 35.0 35.0 5.0

Geography and Environmental Studies

5.0 0 10.0 45.0 35.0 10.0

History

4.5 0 0 45.0 50.0 5.0

Units of Assessment for REF2014Main Panel

Potential Units of Assessment Current REF Lead

A 3. Allied Health Professions, Dentistry, Nursing and Pharmacy Professor Dominic Upton

A 4. Psychology, Psychiatry and Neuroscience Professor Dominic Upton

A 5. Biological Sciences Professor John Newbury

B 11. Computer Science and Informatics Professor Howard Cox

C 17. Geography, Environmental Studies and Archaeology Professor Nick Evans

C 19. Business and Management Studies Professor Howard Cox

C 25. Education Dr Richard Woolley

C 26. Sport and Exercise Sciences, Leisure and Tourism Professor Derek Peters

D 29. English Language and Literature Professor Antonia Payne

D 30. History Professor Antonia Payne

D 34. Art and Design: History, Practice and Theory Professor Antonia Payne

D 35. Music, Drama, Dance and Performing Arts Professor Antonia Payne

UW Preparation Submission being led by Deputy Vice Chancellor and Director of

Research Development (Dr John-Paul Wilson) REF Working Group established 2011 (Institutes, Personnel, ASU) Mock exercise has been undertaken on outputs to identify potential

Units of Assessment Code of Practice for Submission of Staff (for 31st July 2012) Impact Case Study workshops (June/July 2012) Equality Impact Assessments (EIAs) undertaken at each stage Reports to Vice Chancellor’s Advisory Group, Research and

Knowledge Transfer Committee, Academic Board

NOW All outputs should have been put on the WRAP Collect outputs and update the record of outputs New staff will be asked whether they intend to submit Survey of staff circumstances Champions for each UoA to be finalised Preparation of Impact Case Studies Drafting of Research Environment templates Preparation for Survey of Submission Intentions (5th December) REF Submission Pilot

Outputs Submitted staff will need to have produced 4 outputs (publications and

other assessable items) since January 2008 unless they have special individual circumstances

Judgements need to be made about the likely quality of these outputs and whether they adhere to the definition of research in the REF

Some external advice is being sought Co-authorship is allowed Citation data will be used to assess ‘at the margins’ in some UoAs Double weighting of items can be argued

What is an ‘output’? Product of research “process of investigation leading to new insights,

effectively shared” (see handout) First brought into public domain during period 01/01/08 – 31/12/13 By a member of staff (i.e. not a research student) Can include:

printed academic work; new materials, devices, images, artefacts, products and buildings; confidential or technical reports; intellectual property including patents and other forms; performances, exhibits or events; work published in non-print media; reviews, textbooks or edited works (if they embody research)

Impact ‘an effect on, change or benefit to the economy, society, culture, public

policy or services, health, the environment or quality of life, beyond academia’

Impact template (20% weighting) sets out the general approach to supporting impact from research

Impact case studies (80% weighting) includes examples of impacts with evidence of impacts already achieved

Impact relates to the period of the REF (2008-2013) and the underpinning research (of at least 2* quality) must have been undertaken since 1993

1 Case study per 10 staff ftes +1 extra per UoA

Environment Environment data will include:

• Research doctoral degrees awarded• Research income• Research income in kind

Qualitative information on the environment in the University and the Unit of Assessment. The narrative will include:

• Overview of research environment• Research strategy• People (staffing strategy, staff development, research students)• Income, infrastructure and facilities• Collaboration and contribution to the discipline

Code of Practice Submitted in July 2012 (available on the Research Portal) Underpinned by principles of fairness and transparency REF Equality and Diversity Panel will examine all CoPs CoP will be published alongside our submission at end of REF CoP covers: purpose; principles; legal framework; roles & responsibilities;

selection; disclosure of individual circumstances; feedback and appeals Clearly defined circumstances (ECR; PT working; maternity, paternity or

adoption leave; secondments outside the HEI sector) Complex circumstances (disability; ill-health or injury; mental health

conditions; constraints related to pregnancy or maternity; childcare or caring responsibilities; gender reassignment; Other circumstances relating to the above)

Submission System Pilot submission system open now until November 2012 Full submission system will open from 1st January 2013 until 29th

November 2013. JPW and RF have login permissions – inputting can be extended to

others in the institution (maybe academic or administrative staff) Full manual is available from the REF website: http://www.ref.ac.uk/ Research portal will be used to store useful information (including this

presentation and the Code of Practice) under ‘Research Excellence Framework’

http://www.worcester.ac.uk/researchportal/

Audit and Results Audit and verification

• All processes will be audited• All institutions will be audited against at least one element

Results:• An overall quality profile (in 1% steps) will be published for each UoA

against each starred level • Institution will also get a commentary for each UoA• Results will be published December 2014• Publication of submissions, panel overview reports and sub-profiles in

Spring 2015

ANY QUESTIONS?