quality improvement and performance framework - … · · 2016-10-20quality improvement and...

Quality Improvement and Performance Framework

Education Providers

Developing people for health and healthcare

[email protected]@NHS_HealthEdEng

Health Education East of England

2 Quality Improvement and Performance Framework

ContentsIntroduction ................................................. Page 6

Summary of Process ................................... Page 12

Key Performance Indicators ........................ 0

Education Provider RAG Ratings .............................................. Page 24

Annexes ................................................... Page 27

Annex 1 Assessment Surveys ...................... Page 28

Annex 2 Student Surveys ........................... Page 30

Annex 3 Education Provider Key Performance Indicators ........................ Page 32

Annex 4 Guidance for RAG Ratings .............................................. Page 48

Annex 5 Dispute Resolution ....................... Page 50

Annex 6 Aggregation and Rounding .......... Page 52

3

ForewordProfessor Simon Gregory

Director of Education and Quality


Welcome to the Quality Improvement and Performance Framework Handbook which outlines the process for Education Providers (EPs) for 2014/15. I hope that you find it a useful guide for this year.

The purpose of Health Education East of England (HEEoE) is to enhance quality services for patients by ensuring the workforce is planned, educated and trained to a high quality. The Quality Improvement and Performance Framework (QIPF) is the process by which HEEoE assures the education it commissions and delivers on behalf of Employers providing NHS commissioned care in the east of England. The Framework was launched on 01 April 2014, and brings together a number of established processes to assure and continually improve the quality of healthcare education in the east of England. This Handbook focuses on the process for reviewing EPs delivering commissioned education but is designed to be read by all stakeholders in the process, including Employers.

HEEoE invests £377 million every year on commissioning a wide range of education on behalf of local and national health systems. It has a duty to ensure that the EPs delivering this education provide a high standard of credible and professional education.

5

GlossaryARM Annual Review Meeting

DBS Disclosure and Barring Service

Employer Organisation hosting students for practice placements (referred to contractually as ‘Practice Placement Provider’)

EP Education Provider

HEC Healthcare Education Contract

HEEoE Health Education East of England

HSBs Health Service Bodies

KPI Key Performance Indicator

LDA Learning and Development Agreement

MPET Multi Professional Education and Training Levy

OH Occupational Health

QIPF Quality Improvement and Performance Framework

WP Workforce Partnership

Introduction


7

This Handbook provides an overview of the Quality Improvement and Performance Framework for Education Providers (EPs) and gives detailed guidance for organisations involved in delivery of the Framework.

The Quality Improvement and Performance Framework

The Quality Improvement and Performance Framework (QIPF) sets out HEEoE’s approach to ensuring that the quality of the non-medical healthcare education commissioned via Healthcare Education Contracts (HECs) is of high quality. It provides a framework that supports world class commissioning, continually drives up quality, links payment with performance and gives assurance that education and training programmes equip staff with the values, knowledge and skills to deliver high quality care.

QIPF is transparent, locally standardised and actively promotes partnership working between EPs and providers of NHS commissioned services.

This Handbook provides details of process for EPs; a separate but aligned process is available for Employers reviewed as part of QIPF.

The Key Elements of QIPF

There are four key elements to the EP QIPF process:

1. The ongoing collection and analysis of information to support management of EPs

2. The annual triangulation of information provided by all stakeholders involved in education and training

3. Annual Review of EPs against agreed Key Performance Indicators (KPIs)

4. The development and review of plans to support continuous improvement in performance and quality.

Intr

od

uct

ion


Period of Review

The period reviewed as part of QIPF is outlined below:

01 September 2013 to 31 March 2015

Evidence relating only to this period will be reviewed as part of the QIPF process, unless stated otherwise. Key issues relating to performance and quality identified outside of the period may be considered if this is deemed to impact on the Review.

In subsequent years this will be aligned with the NHS financial year April to March.

Intr

od

uct

ion

9

Stakeholder Responsibilities

Employers

n Provide high quality educational learning environments for all students;

n Ensure robust processes are in place to supervise and support student development;

n Actively participate in QIPF, including the Annual Review process;

n Work with EP partners to ensure the continuous improvement of the clinical learning environment and education programmes;

n Utilise the opportunity provided by QIPF to give objective and constructive feedback to EP partners;

n Work with EP partners and Workforce Partnerships (WPs) to use Improvement Plans to make continuous quality improvements.

Education Providers

n Deliver high quality education that meets commissioning requirements and professional standards/values;

n Ensure high quality data is submitted in a timely manner;

n Actively participate in the QIPF, including the Annual Review process as agreed within the pre-registration contracts;

n Utilise Improvement Plans to make continuous quality improvements, working closely with WPs and employers;

n Ensure all identified students are aware of their responsibilities to complete student feedback;

n Undertake an objective and reflective self assessment against agreed KPIs;

n Utilise Improvement Plans to make continuous quality improvements, working closely with WPs and other partners.


Health Education East of England

n Leads the development and improvement of QIPF with key stakeholders;

n Ensures consistency and transparency of approach;

n Manages the Annual Review processes, including the membership of the Review Panel (the Panel);

n Benchmarks outcomes across the region and shares best practice;

n Collects and collates performance data to inform the QIPF;

n Supports the development of capacity and capability within locally devolved systems to manage performance and quality improvement;

n Works with locally devolved systems to deliver year on year improvements in education.

Via Workforce Partnerships:

n Manages EPs’ performance;

n Uses data to review performance;

n Takes a leading role in the Annual Review processes;

n Develops and manages Improvement Plans to deliver ongoing quality improvement and manage risk;

n Provides assurance for commissioned programmes and escalating concerns;

n Supports EPs and Employers to complete constructive, reflective assessment of partners and self assessments.

11

QIPF Outcomes

The aim of QIPF for EPs is to demonstrate and support high quality education outcomes to improve patient care and continuously improve the education being delivered.

Each EP will receive an Annual Report detailing the outcomes of the review process, identifying areas for improvement as well as areas of good practice. The review process is shown in Figure 1. Each commissioned programme will be reviewed against the agreed KPIs and an overall Red/Amber/Green (RAG) rating assigned. The Annual Report and final outcomes will form the basis of the development and Improvement Plans, which will be jointly delivered by the EP, WP and Employers to improve performance and quality in identified areas, and drive continuous improvement.

Intr

od

uct

ion

Measure

Assure

Improve Triangulate

ReviewRate

Figure 1 The Quality Improvement and Performance Framework review process

Summary of Process


Sum

mar

y o

f Pr

oce

ss

13

Student Feedback: HEEoE provides link to Student Surveys to EPs and Employers (see also Annex 2: Student Surveys)

When: September 2014

Student Surveys will be issued in September 2014. These will be for completion by all identified students. Further information on Student Surveys will be issued as an addendum to this document. Closing date will be confirmed in summer 2014 as outlined in Annex 2.

1. HEEoE issues advance copies of EP Self Assessments and Employer Assessments of Partners to EPs and Employers

When: 26 January 2015

Advance copies of the assessments and self assessments will be issued to support both Employers and EPs with the completion of feedback.

2. HEEoE Issues EP Self Assessments and Employer Assessments of Partners to EPs and Employers (see also Annex 1: Assessment Surveys)

When: 16 February 2015

Two surveys will be issued by HEEoE which will form the basis of the qualitative KPIs:

Pre-Registration Student Survey

EPs will receive a link to a pre-registration student survey.

EPs are responsible for ensuring all identified students are aware of their responsibility to provide feedback on their NHS-funded education as defined in the education contracts, and are responsible for issuing surveys to all students currently in training at the EP on programmes commissioned via the pre-registration contract.

The timeline for the process is set out in Figure 2 at the end of this section.


Self Assessment Survey

Assessment of Partners

EPs will receive an academic self assessment survey covering the ten qualitative academic KPIs.

These surveys should be completed with a Self Assessed RAG rating based on the level of evidence the organisation is able to provide, demonstrating that they have complied with the criteria outlined in the KPI. Strong consideration should be given to the narrative provided by all organisations as this will be reviewed as part of the triangulation process (see stage 6 below) with HEEoE and the WPs. Comments are mandatory.

Employers will receive an assessment of EP partners survey covering the ten qualitative academic KPIs.

These surveys should be completed with RAG ratings for each EP from which the employer has hosted students for placement. It is the Employer’s responsibility to ensure the RAG rating and supporting narrative are objective and reflective as this information will be used as evidence in assessing the EP’s performance. It is a requirement that the assessment is signed off by the agreed Director with responsibility for education governance, as outlined in the Learning and Development Agreement (LDA) on behalf of the organisation. Comments are mandatory.

3. EPs complete self assessment

When: 16 February 2015 to 02 March 2015

EP self assessments will be due no later than 13:00 on 02 March 2015. However, the assessment can be submitted at any point between 16 February 2015 and 02 March 2015.

4. Employers complete assessment of EP partners

When: 16 February 2015 to 02 March 2015

Employers’ assessments of EP partners will be due no later than 13:00 on 02 March 2015. However, the assessment can be submitted at any point between 16 February 2015 and 02 March 2015.

Sum

mar

y o

f Pr

oce

ss

15

5. Production of scorecards and reports for Review Panel

When: 16 March 2015

Scorecards are produced for each EP and made available to the Review Panel.

6. HEEoE triangulates all information submitted for discussion at the first Review Panel pre ARM meeting

When: 30 March 2015

The Panel will review the EP self assessments, Employer assessments and progress with the previous year’s Action Plan to allocate tentative RAG ratings for each KPI for each programme being reviewed.

At this stage, where the Panel determines that additional clarification is required, this will be requested from the EP.

7. HEEoE requests additional evidence from EPs where required

When: 30 March 2015

Additional evidence requests issued to EPs. Additional evidence must be provided no later than 09 April 2015.

8. Panel Pre-Annual Review meeting

When: 13 April 2015

Additional evidence is reviewed and considered by the Panel. Key lines of enquiry (KLEs) are agreed by the Panel in preparation for the Annual Review Meeting (ARM).

9. HEEoE issues to EPs key lines of enquiry, draft RAG ratings and composition of Panel for ARMs

When: 14 April 2015

Key lines of enquiry are issued to EP one week prior to the date of its ARM, along with draft RAG ratings for each KPI and confirmation of composition of the Panel for the Meeting. Normally only KPIs where there are outstanding issues will be subject to KLEs at the ARM.


10. HEEoE processes Quantitative Information for Annual Review

When: 20 April 2015

Quantitative KPIs are calculated based on the contract data submitted for 2014/15 Quarter 4. Failure to submit an accurate data return by the agreed deadline will result in all KPIs being rated Red.

11. ARM

When: 20 April 2015 onwards

EP ARMs will focus on KLEs developed in stage 8. Membership of the Annual Review Panel will be confirmed prior to the ARM and will include:

Review Chair, Key members of the Directorate of Education and Quality & Education Commissioning Team, WP Representation, and external members including subject experts and student representation.

For the 2015 Review, the Chair will be the HEEoE Director of Education and Quality. There is an expectation that each EP Dean, with appropriate senior colleagues, will attend their EP ARM. EPs are responsible for ensuring that an appropriate range of Employer partners is present at the ARM. EPs are encouraged to secure representation from the student body and service users as appropriate. When deciding on representation at the ARM, EPs should consider the overall size of the group to ensure it is practical to chair.

12. Annual Report produced and agreed

When: May 2015

An Annual Report for each EP will be produced by HEEoE following the ARM.

The Annual Report will be issued in draft form for the EP being reviewed, within four weeks of its ARM, to agree its accuracy. Final reports will be issued in May.

Sum

mar

y o

f Pr

oce

ss

17

13. Improvement Plan produced, agreed and managed in year

When: May 2015

Improvement Plans should be developed, based on the recommendations outlined in the Annual Report, for delivery in the ten months from 01 June 2015 to 31 March 2016. Responsibility for ongoing management of the Improvement Plan will sit with the WP.

Delivery against agreed Improvement Plans will be assessed as part of the following year’s Annual Review process.


Figure 2 Process Timeline

16Feb2015

02 March 2015

16 March 2015

2 Weeks 1 Week 2 Weeks

Surveys with EPs and

Employers

Panel reviews reports

HEE

oE is

sues

EP

asse

ssm

ents

an

d pa

rtne

r as

sess

men

ts

Dea

dlin

e fo

r as

sess

men

ts a

ndpa

rtne

r as

sess

men

ts

Repo

rts

prod

uced

and

issu

ed t

o th

eRe

view

Pan

el

Student feedback reports provided

26Jan

2015 K

PI q

uest

ions

issu

ed in

adv

ance

inW

ord

docu

men

t

Sum

mar

y o

f Pr

oce

ss

19

30 March 2015

13 April 2015

20 April 2015

2 Weeks 1 Week

Evidence requested KLEs to EPs

Firs

t Pa

nel P

re A

RM M

eetin

g

Seco

nd P

anel

Pre

ARM

Mee

ting

ARM

s,

20 A

pril

onw

ards

May 2015

Out

com

es f

rom

ARM

sag

reed

Quantitative KPIs calculated

Improvement Plans from previous review provided by WPs

Key Performance Indicators


Key

Per

form

ance

Ind

icat

ors

21

This section provides a summary of the KPIs used for the EP QIPF process. Additional information for KPIs can be found in Annex 3. Information in relation to how both EPs and Employers should assign RAG ratings against KPIs is outlined in Annex 4.

Education Provider Key Performance Indicators

EPs will be reviewed against the following KPIs:

R1 Recruitment (Qualitative)

A representative sample of senior staff from Health Service Bodies (HSBs) carries out a stock-take of recruitment and selection policy and processes, and agrees any actions with the EP annually.

R2 Recruitment (Quantitative)

Variance between commissioned numbers and actual students recruited per programme (percentage). Number of starters/number of students commissioned.

A1 Course Content A representative sample of senior staff from HSBs mutually stock takes, annually reviews and agrees with the EP action required to ensure that course content and delivery is suitable for ensuring a workforce that is fit for purpose.

Assurance should also be provided that curriculum content reflects NHS behaviours, values and attitudes required by healthcare professionals as defined by the NHS Constitution.

P1 Partnerships The EP can assure HEEoE that it is able to effectively manage in partnership with HSBs all risks identified within practice proactively therefore minimising the impact on student learning.

P2 Placement Audit A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that the EP places students within currently audited, appropriately staffed clinical areas. The EP ensures that staff supporting students have undertaken appropriate training, and offers updating and link lecturer support to practice.

P3 Fitness for Placement

A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that any concerns about the fitness for placement of students are being responded to in line with processes and timeframes mutually agreed by the EP and the Practice Placement Provider and that the Practice Placement Provider and the EP work in partnership to resolve any issues.


P4 DBS/OH A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that Disclosure and Barring Service (DBS) and Occupational Health (OH) checks and any resultant actions are carried out by the EP in accordance with mutually agreed processes and communicated to Practice Placement Providers appropriately.

P5 Basic Skills A representative sample of senior staff from HSBs confirms that students starting placements demonstrate basic skills, knowledge and professional behaviours as mutually agreed with the EP.

C1 Review Outcomes EP confirms that over the course of the year it has reported any weakness identified by relevant reviews (e.g. Quality Assurance Agency or internal reviews) within two weeks of verbal feedback or as soon as possible, and in any case within three working days of receipt of the written report; whichever is soonest.

In addition, the EP is able to confirm that an action plan has been or is being developed in partnership with Practice Placement Providers or the EP is able to confirm that no weakness was identified by any form of review over the previous year.

C2 Learner Feedback The EP collects student feedback through the National Student Survey and other appropriate means including the QIPF student survey(s) and can demonstrate an audit trail showing resultant action plans and service improvements.

L2 Destinations The EP can assure HEEoE that it is collecting robust information in relation to the first destinations of students, and is using this information appropriately to develop programmes.

O1 Attrition Attrition as a percentage for the programme.

Sum of all (Discontinuances + Withdrawals + External Transfers Out + Internal Transfers Out – External Transfers In – Internal Transfers In) / Sum of all Starters.

23

Key

Per

form

ance

Ind

icat

orsL3 Outturn The percentage of completions on time from the programme

against starters.

= number of students that complete on time / number of starters.

L4 Standard Progression

The percentage of completers on standard progression against overall completers.

= sum of all completers on standard progression on time / sum of all completers.

Education Provider RAG Ratings


Edu

cati

on

Pro

vid

er

RA

G R

atin

gs

25

Each KPI will attract a RAG rating which will provide a total score for each KPI. This score will be multiplied by the weighting assigned for each KPI, which can be found in Annex 3. The RAG ratings are given in the table below:

RAG Description Score

Red Indicates a KPI has not been met or no assurance can be provided of –1 compliance

Amber Indicates a KPI has been partially met 0

Green Indicates a KPI has been met providing a robust level of assurance 1

For each EP there is a maximum score of 100 if all KPIs are rated Green, and a minimum score of –100 if all KPIs are rated Red.

Overall RAG Boundaries

Scores for each of the fourteen individual KPIs will be combined to give an overall score. Overall RAG boundaries for combined KPIs can be seen in the table below:

There are scenarios in which particular KPIs cannot be measured. In these cases, the maximum quantitative score will be 15, 10 or 5, and the overall RAG boundaries will be adjusted to compensate proportionally for the reduced maximum score.

Example 1 – A new programme which has not been running long enough to count towards attrition or have any completers. KPI R2 is measured; O1, L3, L4 are not measured. Maximum quantitative score 5, total maximum score 85.

RAG Score Range

Red Less than 0

Amber Less than 50

Green Greater than or equal to 50


Example 2 – A programme is no longer commissioned but still has cohorts running through the system. KPI R2 is not measured.

n The last cohort completed in September 2014. O1 is not measured, L3 and L4 are measured. Maximum quantitative score 10, total maximum score 90.

n There are cohorts active in/after Jan 2015. O1, L3, L4 are measured. Maximum quantitative score 15, total maximum score 95.

The minimum score is the same value as the maximum, i.e. a programme with a maximum score of 90 will have a minimum score of –90, as set out in the table below:

RAG Maximum 95 (One quantitative KPI not measured)

Maximum 90 (Two quantitative KPIs not measured)

Maximum 85 (Three quantitative KPIs not measured)

Red Less than 0 Less than 0 Less than 0

Amber Less than 48 Less than 45 Less than 43

Green Greater than or equal to 48 Greater than or equal to 45 Greater than or equal to 43

Annexes

An

nex

es

27

Annex 1: Assessment Surveys


29

Both the EP Self-Assessment and Employer Assessment of EPs will be completed online. A link and specific information on how to access and complete surveys will be issued to a nominated contact in each organisation when surveys are issued. All questions in assessment surveys will be based on the qualitative KPIs being measured.

An advance copy of all questions will be provided in Word format on 26 January 2015 to support completion by stakeholders; however, final submissions must be made using the online submission.

Guidance on assigning RAG Ratings

Evidence requirements for each KPI should be noted as outlined in Annex 3. Guidance on how RAG ratings will be assigned against evidence requirements is outlined in Annex 4.

It is important that as much relevant information as possible is provided as part of the assessment surveys to support RAG ratings so as to ensure the Panel is able to consider all factors.

EP Surveys

Assessments and Self Assessments for EPs must provide a RAG rating for each programme offered by the EP. Programmes are rated individually, with one comment against each KPI for all programmes. Where programmes have not been rated Green, reference should be made in the commentary as to why this is the case.

Comments are mandatory against all KPIs; Employers and EPs must be able to provide evidence to support any comments made. This evidence can be in the form of emails, minutes of meetings or any other suitable form which justifies the comments.

An

nex

1A

sses

smen

t Su

rvey

s

Annex 2: Student Surveys


To Follow

This information will be issued as an addendum in Summer 2014.

An

nex

2St

ud

ent

Surv

eys

31

Annex 3: Education Provider KPIs


33

An

nex

3Ed

uca

tio

n P

rovi

der

KPI

s

R1 Recruitment (Qualitative)

A representative sample of senior staff from HSBs carry out a stock-take of recruitment and selection policy and processes and agrees any actions with the EP annually.

Exemplar Evidence

Evidence of a mutually agreed recruitment and selection plan between EP and HSBs showing actions and the outcomes of these actions. As a minimum this plan must include the following topics:

n support for widening access;

n promotion of equality and diversity;

n ensuring candidates’ compatibility with the values and behaviours defined within the NHS Constitution;

n management of Employer concerns in relation to candidates’ suitability;

n clear record of interview decisions is being maintained;

n active engagement of representatives of all Practice Placement Providers in recruitment and selection process;

n active engagement of service users in the recruitment and selection processes;

n active engagement of students in the recruitment and selection process;

n demonstration of innovation and adoption of best practice in the development of marketing, recruitment and selection plans;

n evidence that the EP has ensured that students understand their responsibility as NHS-funded students to provide feedback on their education experience;

n demonstration of partnership working;

n students are recruited in line with Schedule 13 of this agreement;

n plans for ensuring that students are eligible for NHS funding;

n evidence that all staff, including administrative staff, involved in the recruitment and selection of students are aware of the requirement to recruit to the values of the NHS Constitution;

n a mutually agreed process for communicating and managing DBS checks;

n the EP works in partnership with the WP to manage over- and under-recruitment.

EP Assessment Requirements

n The EP is able to provide evidence that it is fully compliant with all exemplar evidence identified.

Employer Assessment Requirements

n The Employer is assured that the EP is fully compliant with the exemplar evidence identified.

This KPI has an overall weighting of 8.


R2 Recruitment (Quantitative)

Variance between commissioned numbers and actual students recruited per programme (percentage).

This is a quantitative measure based on the contract data collection; no self-assessment or assessment by partners is required.

Recruitment will be measured at 31 March 2015 and will cover all new starters reported during the NHS financial year 01 April 2014 to 31 March 2015. Recruitment will be measured against agreed commissioning plans.

The recruitment KPI is a measure of the percentage extent to which the commissioned places have been achieved through recruitment and is calculated as follows:

number of starters

number of students commissioned

For the purpose of recruitment, starters are defined as those students who:

n are first-time entrants to Year 1; and

n have not transferred from another healthcare programme.

Information on aggregation and rounding is outlined in Annex 6.

Programmes will be RAG rated as outlined below:


Red Amber Green

Programmes with commissions of <85% or >110% <95% or >105% 95% to 105% 20 students or more

Small Cohort Calculation: 3 or more students 2 students below Less than or Programmes with commissions of below target OR 3 target OR 2 equal to 1 fewer than 20 students or more students student above student above above target target target or less than or equal to 1 student below target

35

An

nex

3Ed

uca

tio

n P

rovi

der

KPI

s

A1 Course Content

A representative sample of senior staff from HSBs mutually stock-takes, annually reviews and agrees with the EP action required to ensure that course content and delivery is suitable for ensuring a workforce that is fit for purpose.

Assurance should also be provided that curriculum content reflects NHS behaviours, values and attitudes required by healthcare professionals as defined by the NHS Constitution.

Exemplar Evidence

Evidence of a mutually agreed plan between EP and HSBs for reviewing course content and delivery showing actions and outcomes of these actions. As a minimum this plan must include:

n clear process for the management of concerns raised by Practice Placement Providers and/or Students about course content and delivery;

n clear process for involving Practice Placement Providers in curriculum design and delivery;

n curriculum content reflects behaviours, knowledge, skills and attitudes required by healthcare staff as defined within the NHS Constitution;

n curriculum content and delivery are being reviewed annually in partnership with Practice Placement Providers;

n demonstration of partnership working in curriculum design and delivery;

n innovation in curriculum design and delivery;

n evidence that academic staff developing and delivering programmes have up-to-date, relevant clinical knowledge;

n evidence that academic staff developing and delivering programmes reflect the values and behaviours of the NHS Constitution;

n evidence that the EP is responding to national and local priorities such as local Transformation Projects.







P1 Partnerships

The EP can assure HEEoE that it is able to manage all risks identified within practice effectively and proactively in partnership, thereby minimising the impact on student learning.

Exemplar Evidence

Evidence must be available on request from a representative sample of senior staff from HSBs confirming the KPI is being met.

Acceptable evidence would be:

n minutes of meetings containing clear and unambiguous reference to the confirmation that all risks are reviewed annually and that there are no risks or that risks are being managed effectively;

n where risks are identified, a clear, mutually agreed action plan is available;

n a signed letter from representative senior practice staff from HSBs agreeing that the KPI is being met (as above) in partnership with all stakeholders;

n Operational Contract Meetings are being held regularly with representation from the EP, practice placement partners and the WP;

n Strategic Review Meetings are effective and resolve those operational issues which are escalated.






37

An

nex

3Ed

uca

tio

n P

rovi

der

KPI

s

P2 Placement Audit

A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that the EP places students within currently audited and appropriately staffed clinical areas. The EP ensures that staff supporting students have undertaken appropriate training, and offers updating and link lecturer support to practice.

Exemplar Evidence

Evidence must be available on request from a representative sample of senior staff from HSBs that confirms the KPI is being met.


n minutes of meetings containing clear, unambiguous reference to the confirmation;

n a signed letter from representative senior practice staff from the HSB agreeing that the KPI is being met in partnership with all stakeholders;

n the EP is working with local HSBs to ensure that current and future placement circuits reflect the changing NHS and can provide documentation demonstrating this.







P3 Fitness for Placement

A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers, confirms that any concerns about the fitness for placement of students are being responded to in line with processes and timeframes mutually agreed by the EP and the Practice Placement Provider and that the Practice Placement Provider and the EP work in partnership to resolve any issues.

Exemplar Evidence

Evidence must be available on request from a representative sample of senior staff from HSBs that confirms the KPI is being met.


n minutes of meetings containing clear, unambiguous reference to the confirmation in partnership with all stakeholders;

n a signed letter from representative senior staff from the HSB agreeing that the KPI is being met.

A clear policy, mutually agreed and implemented by the EP and HSB, should exist regarding fitness for placement including:

n cause for concern processes implemented and effective;

n reporting of untoward incidents that involve students to the EP and HEEoE;

n systems in place for monitoring individual students as they progress through each practice placement;

n capacity of each practice placement to support students is being managed consistently;

n placement audit systems that record capacity and are reviewed annually.






39

An

nex

3Ed

uca

tio

n P

rovi

der

KPI

s

P4 Disclosure and Barring Service and Occupational Health checks

A representative sample of senior staff from HSBs, as agreed with Practice Placement Providers confirms that Disclosure and Barring Service (DBS) and Occupational Health (OH) checks and any resultant actions have been carried out by the EP in accordance with mutually agreed processes and communicated to Practice Placement Providers appropriately.

Exemplar Evidence

Evidence must be available on request from a representative sample of senior staff from providers of NHS services that confirms the KPI is being met.


n evidence of 100% compliance with the policy; no students ever enter placement without the necessary DBS and OH clearance;

n evidence that DBS and OH clearance is communicated to all Practice Placement Providers in advance of students entering placements. For clarity, reporting should not be by exception;

n minutes of meetings containing clear, unambiguous reference to confirmation the KPI is being met in partnership with all stakeholders;

n a signed letter from representative senior practice staff from the HSB agreeing that the KPI is being met.







P5 Basic Skills

A representative sample of senior staff from HSBs confirms that students starting placements demonstrate basic skills, knowledge and professional behaviours as mutually agreed with the EP.

Exemplar Evidence

Evidence must be available on request from a representative sample of senior staff from HSB that confirms the KPI is being met.


n minutes of meetings containing clear, unambiguous reference to confirmation the KPI is being met in partnership with all stakeholders;

n a signed letter from representative senior practice staff from the HSB agreeing that the KPI is being met;

n students demonstrate basic skills, knowledge and behaviours which reflect professional codes of conduct and the values of the NHS Constitution;

n students complete all mandatory training prior to commencement in placement;

n students have a clear induction from service staff prior to commencing each placement;

n record of induction held as part of the placement documentation. Students have clear placement guidelines and expectations;

n guidelines and expectations agreed between EP and HSB and mechanisms are in place to ensure patients are aware and consent to care being provided by students;

n ensuring student awareness of a positive attitude to hearing, accepting and responding to student concerns, complaints and compliments.






41

An

nex

3Ed

uca

tio

n P

rovi

der

KPI

s

C1 Review Outcomes

EP confirms that over the course of the year it has reported any weakness identified by relevant reviews (e.g. QAA or internal reviews) within two weeks of verbal feedback, or as soon as possible and, in any case, within three working days of receipt of the written report; whichever is soonest.

In addition, the EP is able to confirm that an action plan has been or is being developed in partnership with Practice Placement Providers, or the EP is able to confirm that no weakness was identified by any form of review over the previous year.

Exemplar Evidence

n Reports from other reviews should be available if requested;

n A signed letter from the Dean agreeing that the KPI is being met;

n Action plans developed in partnership with HSBs.







C2 Learner Feedback

The EP collects student feedback through the National Student Survey and other appropriate means including the QIPF student survey(s) and can demonstrate an audit trail showing resultant action plans and service improvements.

Exemplar Evidence

Action plans and collated student feedback for each cohort must be available on request and should cover the following topics as a minimum:

n course content and delivery;

n adequacy of preparation for placements;

n placement learning experiences;

n service improvements.

Evidence should be available to demonstrate:

n how the EP is increasing response rates;

n an effective approach to interrogating the data;

n effective mechanisms for feeding back to students;

n process for making changes based on feedback;

n how the EP is measuring effectiveness of changes made as a result of student feedback.






43

An

nex

3Ed

uca

tio

n P

rovi

der

KPI

s

L2 Destinations

The EP can assure HEEoE that it is collecting robust information in relation to the first destination of each student, and is using this information appropriately to develop programmes.

Exemplar Evidence

The EP should provide evidence that it is:

n making all reasonable efforts to collect information on students’ first destination after qualifying;

n attempting to obtain information on first destinations from all qualifying students;

n encouraging and supporting students to enter employment in healthcare within the east of England;

n working in partnership with organisations providing NHS-funded care in relation to the recruitment of newly qualified healthcare professionals into the workforce.







O1 Attrition (Quantitative)

Attrition as a percentage for the programme.


Attrition is measured at 31 March 2015 and includes cohorts which were active during the Quarter 4 reporting period (i.e. have a completion date on or after 1 January 2015) and have been active for at least three months (i.e. commence on or before 31 December 2014).

The KPI value is produced for each programme by grouping equivalent eligible cohorts (those which have a common subject or nursing branch, level, and duration), and calculating the aggregate rate of attrition over all cohorts.

The percentage attrition for a programme, based on the eligible cohorts, is given by the formula:

Sum of all (Discontinuances + Withdrawals + External Transfers Out + Internal Transfers Out – External Transfers In – Internal Transfers In) / Sum of all Starters

Definitions of the items used in the formula:

Discontinuation: A student who leaves by the EP decision

Withdrawal: A student who leaves by their own decision

External Transfer: A student who moves to or from a different EP, with credit

Internal Transfer: A student who moves between programmes or cohorts within an EP, e.g. a change of nursing branch or level, or returning after a break in study and moving to a later cohort

Starters: For the purpose of attrition, starters are defined as those students who:


n have not transferred in from some other healthcare programme.


RAG rating tolerances are outlined over the page for all programmes.


45

For programmes with 8 or more students:

Programme Red Amber Green

Adult Nursing More than 20% 13% to 20% 13% or Less

Children’s Nursing More than 20% 13% to 20% 13% or Less

Learning Disabilities Nursing More than 30% 13% to 30% 13% or Less

Learning Disabilities Nursing More than 30% 13% to 30% 13% or Less & Social Work

Mental Health Nursing More than 30% 13% to 30% 13% or Less

Mental Health Nursing & Social Work More than 30% 13% to 30% 13% or Less

Midwifery 3-Year More than 18% 13% to 18% 13% or Less

Midwifery 18-Month More than 15% 13% to 15% 13% or Less

Physiotherapy More than 15% 13% to 15% 13% or Less

Occupational Therapy More than 15% 13% to 15% 13% or Less

Speech and Language Therapy More than 15% 13% to 15% 13% or Less

Dietetics More than 15% 13% to 15% 13% or Less

Diagnostic Radiography More than 15% 13% to 15% 13% or Less

Therapeutic Radiography More than 25% 13% to 25% 13% or Less

Operating Department Practice More than 15% 13% to 15% 13% or Less

Clinical Psychology More than 15% 13% to 15% 13% or Less

Health Visiting More than 10% 7% to 10% 7% or Less

Clinical Physiology More than 15% 13% to 15% 13% or Less

Biomedical Science More than 15% 13% to 15% 13% or Less

Oral Health Foundation Degree More than 20% 13% to 20% 13% or Less

Small Cohort Calculation: Red Amber Green For programmes with fewer than 8 students

3 or more net 2 net leavers 0 to 1 net

leavers leavers

An

nex

3Ed

uca

tio

n P

rovi

der

KPI

s


L3 Outturn

The percentage of completions on time from the programme against starters.


This KPI measures the number of students who complete from a programme on time as a percentage of the number of starters for that programme.

n Completion on time is defined by a student having an actual completion date no earlier than 100 days before and no later than 45 calendar days after the programme completion date.

n Completion is defined as a student who qualifies from a programme, which may or may not be the same programme that they originally enrolled on.

n Students who qualify with an exit award that does not make them eligible to register with a professional body will not be classed as completers.

For the purpose of completions starters are defined as those students who:


n have not transferred from another healthcare programme.

Outturn will be measured on 31 March 2015 for all programmes with a completion date between 1 January 2014 and 31 December 2014.

The percentage of students who complete a programme, on time, as a percentage of the number of starters is calculated using the following formula:

number of students that complete on time

number of starters


RAG Rating tolerances for this KPI are outlined in the table below:


Red Amber Green

Programmes with commissions Less than 45% Less than 60% Greater than, or of 4 students or more equal to 60%

47

An

nex

3Ed

uca

tio

n P

rovi

der

KPI

s

L4 Standard Progression

The percentage of completers on standard progression against overall completers.


This KPI measures the number of students who complete on standard progression on time in a calendar year as a percentage of the total number of completers that year.

Standard Progression will be measured at 31 March 2015 for all students with an actual completion date between 01 January 2014 and 31 December 2014 inclusive.

The number of students who complete on standard progression on time in a calendar year as a percentage of the total number of completers that year is calculated using the following formula:

Sum of all completers on standard progression on time

Sum of all completers

Completer on Standard Progression: A student on standard progression completes the programme of study for which they originally enrolled and qualifies with an exit award that makes them eligible to join the professional register, without re-sitting such that they would not complete within the normal timeframe. The normal timeframe includes students whose actual completion date is no earlier than 100 days before and no later than 45 days after the programme completion date. Students who enter a programme as direct entrants with Accreditation of Prior and Experiential Learning (APEL) and who continue and complete on time in that programme are included in the students who complete under standard progression transfers from nursing diploma to degree programmes, or between health visiting programme levels, are not classed as leaving standard progression, unless the cohort year, nursing branch or programme duration is also changed.

Completer: A student who completes a programme of study and qualifies with an exit award that makes them eligible to join the professional register. If this is not the same programme they originally enrolled on, they are not a completer on standard progression.

Completion on time: This is defined by a student having an actual completion date no earlier than 100 days before and no later than 45 calendar days after the programme completion date.

RAG Rating tolerances for this KPI are outlined in the table below:


Red Amber Green

All Programmes Less than 78% Less than 82% Greater than or equal to 82%

Annex 4: Guidance for RAG Ratings


An

nex

4G

uid

ance

fo

r R

AG

Rat

ing

s

49

To ensure a consistent regional approach, all RAG ratings will be subject to peer review and agreement prior to ARMs to ensure consistency within the information being reviewed and the outcomes being agreed. The following guidance has been provided as a tool to support the determination of RAG ratings for each qualitative KPI.

Green Amber Red

Evidence has been provided that all criteria have been met as identified in the KPI.

Assurance has been provided to the extent required in the KPI.

The Review Panel is confident that the subject of the ARMs will be able to maintain performance against the KPI for the period until the next Annual Review.

Evidence has been provided that the subject of the Review is making all appropriate effort to achieve the objectives outlined in each KPI.

Partial evidence can be provided of compliance with a KPI.

Partial assurance has been provided of compliance with a KPI.

No serious concerns have been identified; however, outstanding performance has also not been identified.

While not all elements of the KPI have not been met, the current level of compliance is not deemed to be unsafe, or of a quality which would impact on levels of patient care.

Serious concerns are identified about one or more element included in a KPI.

Evidence cannot be provided of compliance with several elements of a KPI.

The Review Panel is not assured of compliance with several elements of a KPI.

Significant review is required to address issues outlined in the KPI.

Panel is not confident that issues can be addressed within a reasonable timeframe.

Annex 5: Dispute Resolution


An

nex

5D

isp

ute

Res

olu

tio

n

Although a three-step dispute resolution process has been agreed for QIPF, it is expected that Steps 2 and 3 will only be used as a last resort and that the informal resolution process described in Step 1 will be attempted first.

The dispute resolution steps

Step 1 Individual EP concerns should first be raised with the WP. The WP will endeavour to find an informal resolution to the problem through discussion and mediation, involving others as necessary.

Step 2 If the concerns cannot be resolved at this level, the matter should be raised by either party (or both) with HEEoE. The Head of Education and Commissioning may convene an appropriately constituted panel to consider the matter further and appoint a mediator, involving others as necessary.

Step 3 If the EP is still not happy, the last recourse in this local process would be to HEEoE.

51

Annex 6: Aggregation and Rounding


An

nex

6A

gg

reg

atio

n a

nd

Ro

un

din

g

Aggregation

n For nursing, where an EP delivers both diploma and degree-level nursing, programmes will be aggregated. Masters-level nursing programmes will be treated separately.

n Where a programme has multiple intakes within the period covered by the KPI, these will be aggregated for the calculation of KPI values.

n All health visiting programmes will be aggregated based on programme duration, and not based on programme level.

n Where an EP delivers education programmes at different levels or for different durations, other than the programmes identified above, these will be treated as separate programmes when reviewing programmes and calculating KPIs.

Rounding

Where the quantitative KPIs are based on a percentage value, these values will be rounded to two decimal places before the RAG rating is applied.

The scoring boundaries are set such that a value outside a higher band limit falls in to the RAG below:

Red: Less than 85% including all scores from 0% to 84.99%

Amber: 85% to 94% including all scores from 85% to 94.99%

Green: 95% to 100% including all scores from 95% to 100%

Amber: 101% to 110% including all scores from 100.01% to 110%

Red: More than 110% including all scores from 110.01% and above.

53

Contact Information

Professor Simon Gregory Director of Education and Quality

[email protected]

Jenny McGuinnessHead of Education Commissioning

[email protected]

Richard Davies Education Commissioning Manager

[email protected]

Chris BirbeckHead of Quality Improvement

[email protected]

Sue AggerSenior Quality Improvement Manager

[email protected]

Prinect PDF Report 13.00.028 - 1 - 10/09/2014 09:35:34 AM

Document overviewFile name: 86597_HANDBOOK.pdfTitle: -Creator: Adobe InDesign CC (Macintosh)Producer: Adobe PDF Library 11.0Author: -Creation Date: 09/29/2014 11:08:55 AMModification Date: 09/29/2014 11:12:43 AMFile size: 8.8 MByte / 9029.1 KByteTrapped: NoOutput Intent: -PDF/X Version: -PDF Version: 1.5Number of pages: 54Media Box: 232.48 x 259.48 mmTrim Box: 215.90 x 242.90 mm

Summary Error Warning Fixed InfoDocument - - - -PDF/X - - - -Pages - - - 1Colors - - - -Fonts - - - -Images - - - -Content - 53 - 38

PagesNumber of pages in document 54 is greater than 1

ContentTrim missing #53 (2-54)Transparency is present #38 (1,4,6-8,10,12-14,16,18,20-22,24-34,36,38,40,42,44,46,48-53)

Additional informationColor separations: 4

CMYK

Color spacesDeviceCMYK / Separation

Fonts: 12Frutiger-Black Type1 / WinAnsi / embedded subsetFrutiger-Bold (2x) Type1 / WinAnsi / embedded subsetFrutiger-Bold Type1 / Custom / embedded subsetFrutiger-BoldItalic Type1 / WinAnsi / embedded subsetFrutiger-Light (2x) Type1 / WinAnsi / embedded subsetFrutiger-Light Type1 / Custom / embedded subset

Prinect PDF Report 13.00.028 - 2 - 10/09/2014 09:35:34 AM

Frutiger-Roman Type1 / Custom / embedded subsetHelvetica TrueType / WinAnsi / embedded subsetMinionPro-Regular Type1 / WinAnsi / embedded subsetZapfDingbatsITC TrueType (CID) / Identity-H / embedded subset

quality improvement and performance framework - … · · 2016-10-20quality improvement and...

Documents