essential skills for effective educational supervisioncmft.nhs.uk/media/1385215/essential...
TRANSCRIPT
Essential Skills for Effective
Educational Supervision
Drs John Bright & Margaret Kingston
Associate Directors Post Graduate
Medical Education, CMFT
Recent changes in
workplace based medical education
• National changes to the NHS & impact on training
– Health & Social Care Act
• Health Education England
• Regional changes:
– Health Education North West
• Merger of the old Deaneries
• Trust changes in service & education delivery
– Trafford etc
– Service pressure
• Shape of training
– What is it?
– How, or will it be implemented?
• Changes in educational supervision
– Revalidation & GMC standards
• GMC guidance for the training workplace environment
• Recent changes to WBAs & ePortfolio
• WBAs, writing the
required reports & preparing for ARCPs
Health Education England
• Special Health Authority fully operational from 2013
• Responsible for education, training & workforce planning for all HCPs
• Responsive to clinical need
• Managing the 13 LETBs
• Ours is HENW & for PGME:
• North West Deanery
• Mersey Deanery
•Currently in the process of merging
•Large cost savings
GMC standards for trainers
• Phased introduction from July 2013 -2016
• New supervisors require specific training
• Existing supervisors must undertake annual education CPD and provide evidence for the GMC trainer domains in their portfolios
1. Ensuring safe and effective patient care through training
2. Establishing and maintaining an environment for learning
3. Teaching and facilitating learning
4. Enhancing learning through assessment
5. Supporting and monitoring educational progress
6. Guiding personal and professional development
7. Continuing professional development as an educator
Who implements
these standards?
• GMC set the standards (some LETB interpretation)
• LEP (the trust) have systems in place to accredit trainers & record evidence
• Deanery visits assess the process & sample 10% trainers evidence
– CMFT March 2015:
• Self report (all 65)
• 41 (63%)
• Audit evidence provided (20)
• 11 (55%)
• General comments:
• Work to do re: Knowledge & applicability of the standards….
• Responses given the turnaround time quite good!
Clinical supervisor A named clinical supervisor is a trainer who is
responsible for overseeing a specified trainee’s
clinical work
Provide feedback during that placement
Responsible for patient safety
Fully trained in specific area of clinical care
Offers supervision tailored to needs of the trainee
Ensures trainee is not expected to work beyond
their competence
Educational supervisor A named educational supervisor is responsible
for the overall supervision and management of a trainee’s trajectory of learning and educational progress
The educational supervisor helps the trainee to plan their training and achieve agreed learning outcomes.
He or she is responsible for the educational agreement and for bringing together all relevant evidence to form a summative judgement at the end of the placement or series of placements.
Structure for Trainers:
AoME Standards
1. Ensures safe and effective patient care through training
(Clinical & Educational Supervisor)
2. Establishes and maintaining an environment for learning
(Clinical & Educational Supervisor)
3. Teaches and facilitates learning
(Clinical & Educational Supervisor)
4. Enhances learning through assessment
(Clinical & Educational Supervisor)
5. Supports and monitors educational progress
(Educational Supervisor)
6. Guides personal and professional development
(Educational Supervisor)
7. Continuing professional development as an educator (Clinical & Educational Supervisor)
The Educational Supervision Timeline
• Know who you are expecting (Educational unit lead)
• Ensure New starters have Induction
• First Meeting (2-4 weeks)
• Portofolio Review (beforehand)
• Sign Agreements
• Review Induction and Mandatory training
• The Induction Passport / Medical Devices/ AnTT
• Induction Portal
• Review PDP
• Define Goals and Assessments to be used
• Curriculum Review (understand what they need to achieve)
• ARCP decision Aid
• Set Timetable and ensure colleagues aware
• Ensure Time to Attend Training is provided
• Understand your time and SPA allocation
• Weekly interaction?? / time to gain feedback from others
• Document and provide a copy
The Educational Supervision Timeline
• Midpoint Review Structured Meeting
• Progress to PDP / WBA’s
• Portfolio Engagement and Sampling
• Unmet educational needs / Opportunities
• Review changes to any ARCP guides
• Ensure Time allocated to attend training
• End of Attachment Review
• Review PDP / WBA’s / MCR / MSF / Audit
• Review of Incidents and compliments
• Portfolio Review and Sampling
• Unplanned interactions
• HLI’s
• Untoward events
• Praise and compliments
The Educational Supervision Timeline
• Preparation For ARCP’s ( know when they are )
• Career Guidance / Next stage interviews
(what do they need or have to do to progress )
• Educational Supervisors Report
Be Honest and Truth-full BUT Document and ensure
more than one Source
JRCPTB: Recommendations 2014
AoPs are Rubbish
SLEs will continue to focus on constructive feedback and action plans
The educational supervisor should report on the trainee's engagement with the
curriculum and learning demonstrated through SLEs and other evidence. The
report must also include a summary of feedback multiple consultant report
(MCR) and multi-source feedback (MSF).
Use of ARCP decision aids.
The number of links to curriculum competencies for different SLEs will be limited
and clearly defined.
Trainees may link SLEs and other evidence to curriculum competencies in order to
demonstrate engagement with and exploration of the curriculum.
Supervisors should sample the evidence linked to competencies in the ePortfolio.
Development checklists for group sign off of competencies, on the basis of the
supervisor's deep knowledge of Trainee
Ten of the common competencies will not require linked evidence in the ePortfolio.
\
ENVIRONMENT
Learning Culture in the workplace Developing openness Involving whole team & patient Shared values & understanding
Valuing & including trainees
Individual Trainee Friendly supportive relationship
Appropriate clinical supervision
Ensure named supervisor Maximise learning opportunities
Department Faculty Group Attend meetings Discuss trainees’ performance
Consistency of approach
CPD for educational role
Service Provision Proactively manage workload
Ensure patient safety
Allow trainee to take responsibility
Teach & role-model efficiencies
Thanks to Shirley Remington
WPBA :Workplace based Assessment
‘a systematic procedure for measuring a trainee’s progress or level of achievement, against defined criteria to make a judgement about a trainee' GMC 2010
'Consistent across entire training programme and
which is blueprinted against, and supports, the
approved curriculum'
Competence (can do)
Performance (does)
WPBA :The Rules
Within the Clinical Environment
No Compromise Safety and Quality Care
Structure must be documented in the public domain
Match the curriculum (areas of Good Medical Practice)
Add unique information and build on previous
Knowledge
Skills
Attitudes
Behaviour
WBA: We admit it they are CR*P
• Widespread comments from trainees and supervisors that the WPBA
system was a burdensome process, regarded by many as a tick box
exercise, incapable of detecting poorly-performing trainees.
Incorrect use to determine trainee progression was impairing the
correct use of WPBAs as aids to learning
To overcome poor Reliability:
Multiple Assessments
Multiple Assessors
Triangulation
To Build a Picture
WORKPLACE-BASED ASSESSMENT
• Assessment for learning (formative) = Supervised Learning Events
• Assessment of learning (summative) = Assessments of Progress
• Learning is at its most powerful when it is authentic (workplace)
• Valid but not always reliable assessor (subjective versus objective)
• Reliability when part of many
• Learning by doing, reviewing, reflection
AUTHENTICITY OF CLINICAL ASSESSMENT – MILLER’S PYRAMID
Miller (1990)
Knows
Shows how
Knows how
Does
Pro
fessio
nal au
then
ticit
y
Behaviour
Cognition
Where is the truth (the diagnosis??)
But I am having a bad day
He always marks everyone Low
I’ll never see a case like that again
WHAT DO WE ASSESS?
Clinical
knowledge and
skills
Practical skills
Interpersonal
skills and
judgement
Professional
behaviours
Case-based discussion
(CBD)
Direct observation of
procedures (DOPs)
Direct observations of
non-clinical skills
(DONCS)
360º appraisal (360)
Overview of WBA
What the trainee actually does
Competencies demonstrated ‘when ready’
Assessment of developmental progression - guides
decisions about future learning
Recorded in an electronic portfolio
Process is learner led - trainee has to ensure their
e-portfolio covers the e-curriculum
WBA: The Trainee's Responsibility
To build on previous assessments
To set agenda based on need and
opportunities
To define their learning tools / methods
To promote regular dialogue with
supervisors
Through reflection define self direct learning
All assessments should be discussed with
educational supervisor
Provide feedback on learning environment
WBA: The Trainer's Responsibility
Safety beats educational needs
Job plan PA's and resources
Cooperative structure which is trainee
lead
Awareness of standards to which trainee
assessed
To provide feedback via debriefing and
document feedback
WPBA: Limitations
WPBA does not assess knowledge
directly Opportunistic
Aspiration to excellence can be lost
Requires engagement from educational supervisor
Competitive behaviour
Resource allocation time and renumeration
Everyone wants to be nice
Fear of consequences of negative comments
(both sides)
“Can we do a
Mini-CEX?”
“Of course, just send
me a link”
“one of the most stubborn problems with workplace-based
assessment is observation of trainees, or rather the absence of it” Van Dreissen, Medical Teacher, 2013; 35: 569–574
WORKPLACE-BASED ASSESSMENT
WBA Competencies Examples of Assessors Setting
Mini-CEX
Communication with patient,
physical examination, diagnosis,
treatment plan
Educational/ Clinical Supervisors,
senior trainee
Clinic, A&E, ward,
community
CBD Clinical judgement, clinical
management, reflective practice
Educational/ Clinical Supervisors,
senior trainee
Multiple areas covered by
a challenging case
DOPs Technical skills, procedures
and protocols.
Educational/ Clinical Supervisors,
senior trainee
multi professional team (MPT)
Clinic, A&E, ward, theatre
Mini-PAT
MSF
TAB
Team-working,
professional behaviour Trainee’s MPT
Multiple areas covered by
MPT
PBA/OSAT Technical skills, procedures and
protocols, theatre team-working Consultant or ST5 + trainee Clinic, A&E, ward, theatre
WBA: The Trainer's Responsibility The
Assessment
Review that element of the trainee’s work
Evaluate it against a reference framework that
reflects the pre- set learning objectives and the
level expected at that stage of learning
Make a judgement and provide feedback to the
trainee on that judgement "word descriptors"
Agree with the trainee an educational action
plan to build on that judgement with further
learning
FEEDBACK DOS AND DON’TS
Do Don’t
Do it close to the observation Avoid inappropriate place/time
Describe the specific behaviour Judge the person
Only comment on what you see Generalise
Give examples of what was good and why Break confidentiality
When identifying areas for improvement
suggest alternatives Be vague
Give follow-up actions for development Avoid specifics
GIVING FEEDBACK – REFLECTION
• How do you think it went? (insight check)
• What went well?
• Examples of the good
• What could be improved/how?
• ‘I noticed…’
• ‘If you were doing it again…’ (ask/suggest)
Describe gap between current and desired performance
Agree a plan and how to get there
WHY DO WE ASSESS?
• Patient safety and standards of care
• To ensure we are training correctly
• To develop trainees and monitor progression link to ARCP
• Transparent Process / Blueprint (clear guidance to want is expected)
• Early remedial action if required
TRAINEES SHOULD BE ‘SAFER’
• Spread assessments through job
• As many assessors as possible
• Feedback as well as scores
• Evidence it all (follow-up actions)
• Reflect on what they do
What do you think of WBAs?
What have
you found
works well
with WBAs?
What have
you found
does not
work well
with WBAs?
Personal development plan
• Keep it to a few objectives
• Objectives should be SMART Specific Measurable Agreed and achievable Realistic Time-bound
• Demonstrable (may be harder do SMART eg becoming a more effective team member)
Reports required
Multiple Consultants Report (MCR)
• To inform the
educational
supervisor's report
• Most specialties
require 4 (0-6 incl.
GIM)
• Required from 6
August 2014
Educational Supervisors Report
• Pivotal to the ARCP
process and should refer
to the MCRs and MSF
• Should report on
engagement with the
curriculum determined by
sampling of evidence and
competencies
Educational Supervisors Report
1. Overall assessment of trainee since last ARCP
2. Evidence of progression as expected at that point in training
3. Must have reviewed portfolio and know what is needed for that year of training to sign off that completed requirements
4. Flag up any areas of concern in training & also helps set goals for excellent trainees and identifies their development needs
ARCP Preparation
Trainee
• Complete the required
assessments
• Self-assess competencies the
curriculum items
• PDP for next year and revise
CV
• Reflective pieces where
required
• If relevant: CCT calculator
Ed Supervisor • Check assessments complete
• Complete MSF summary
• Complete supervisor’s report
• Sample 10-12 of the curriculum items in the eportfolio and assess the evidence linked & self-rating
• Review the previous ARCP recommendations and ensure that completion of suggested tasks is obvious
The ARCP –
what do assessors check?
– Profile – where they are & what are they doing
– Progression – last ARCP outcome & PDP
– Curriculum requirements – are they met for that year?
– Assessment – as per the ARCP grid & curriculum
– Reflection – learning points
– Educational supervisors report & provide feedback on this
Recent changes in
workplace based medical education
• National changes to the NHS & impact on training
– Health & Social Care Act
• Health Education England
• Regional changes:
– Health Education North West
• Merger of the old Deaneries
• Trust changes in service & education delivery
– Trafford etc
– Service pressure
• Shape of training
– What is it?
– How, or will it be implemented?
• Changes in educational supervision
– Revalidation & GMC standards
• GMC guidance for the training workplace environment
• Tips for effective clinical & educational supervision
• WBAs, writing the required reports & preparing for ARCPs
Any further questions?