Exploring the

Benefits and Challenges of

Competency Based Medical

Education (CBME) Implementation

Nicolette Caccia MD FRSCS MEd Department of Obstetrics & Gynaecology, University of Toronto

Department of Obstetrics & Gynaecology, McMaster University

Department of Obstetrics & Gynaecology, Queens University

Filomena Meffe MD FRSCS MSc Department of Obstetrics & Gynaecology, University of Toronto

University of Toronto Department of Obstetrics & Gynaecology Faculty Development Day Workshop Nov 11 2015

Learning Objectives

• review the results of the 2015

Departmental CBME Needs Assessment

• examine the perceived benefits and

challenges of CBME implementation

• evaluate these challenges in the various

clinical care areas

• develop suggestions to address these

challenges

Hours per week with residents

< 4 hrs/wk

4-8 hrs/wk

8-16 hrs/wk

> 16 hrs/wk

Demographics Site of Practice

Academic

Community

Both

Time in

Practice

< 3 years

3-5 years

6-10 years

11-20 years

> 20 years

Other

Assessment Techniques

Providing Feedback

Techniques of Direct Observation

I have adequate training in:

Strongly disagree/Disagree

Neutral

Agree/Strongly agree

I have enough time to allow for:

Strongly disagree/Disagree Neutral Agree/Strongly agree

OR L+D Clinic Ward

Observation

Assessment

Feedback

I am familiar with:

Strongly disagree/Disagree Neutral Agree/Strongly agree

CanMEDS (2005) CanMEDS 2015

CBME CBME Ob Gyn is to be implemented by 2019

Assessment of Resident Competence:

Strongly disagree/Disagree

Neutral

Agree/Strongly agree

Able to Assess Competence

Effective Tools to Assess Competence

Have clear idea of competence on starting

Helpful to know trainee’s competence …

Able to identify resident-in-difficulty in a timely fashion

Familiarity with Assessment Tools:

Strongly disagree/Disagree Neutral Agree/Strongly agree

Written Test

Structured Oral Exam

OSCE Standardized

Patients Simulation

Encounter Card

Logbook Portfolio

Mini-CEX

360s OSATS

OSPRE Rubrics Script

Concordance Test

Much/Somewhat worse than present Same as present

Somewhat/Much better than present Don't have enough information

CBME will affect our ability to: Assess

Residents

Educate Residents

Assess Competency

Give Positive Feedback

Identify Resident-in-difficulty In Timely Fashion

Justify Resident Failure

Implementation of CBME will affect: OR L+D Clinic Ward

# Patients Booked

Residents Supervision

Residents Interaction

Strongly Disagree/Disagree Neutral Agree/Strongly Agree Don’t have enough information

Perceived barriers to negative feedback:

Unsure/Not importantSlightly/Somewhat importantVery important/Of ultimate importanceDon't have enough information

Feedback Too Infrequent

Lack Of Time To Give Feedback

Assessment Tools Do Not Have Useful Descriptors

Negative Feedback Won’t Make A Difference

Concern About Resident's Feelings

Concern With Having Difficulty Working With

Resident In Future

Concern About Lower TES

Fear of Personal Repercussions

Perceived Faculty Development Needs:

Unsure/Not important Slightly/Somewhat importantVery important/Of ultimate importance Don't have enough information

Strategies to IDENTIFY

resident-in-difficulty

Giving FORMATIVE FEEDBACK to resident-in-difficulty

Understanding of different ASSESSMENT TOOLS

Understanding of the USE OF ASSESSMENT

TOOLS IN CBME

Perceived Faculty Development Needs:

Unsure/Not important Slightly/Somewhat importantVery important/Of ultimate importance Don't have enough information

Understanding CanMEDS Basics

CanMEDS Medical Expert

CanMEDS Non-Medical Expert

Information about how OTHER OBGYN programs are implementing CBME

Information about how OTHER DISCIPLINES

are implementing CBME

Perceived Faculty Development Needs:

Unsure/Not important Slightly/Somewhat importantVery important/Of ultimate importance Don't have enough information

Understanding how CBME will affect my

CLINICAL PRACTICE

Understanding how CBME will alter my

RESPONSIBILITIES AS A TEACHER

Understanding MY ROLE in the implementation of CBME over

the next 4 years

Understanding how CBME may change the RESIDENCY PROGRAM goals and …

Summary of Findings

• Faculty are aware that: – CBME is coming – time for assessment and feedback will need to be built in – most changes will be in clinic and on the ward

• Faculty willing to give feedback but need time, tools and education to do so

• Perceived Faculty Education Needs – CanMEDS intrinsic competencies – CanMEDS 2015 – Assessment tools – Assessment practices – Feedback techniques

• Faculty want to be kept informed & participate – especially as to how CBME will affect practice

National Challenges • Development of:

– Electronic mobile friendly e-portfolio – Tools to assess CanMEDS 2015 – Bank of assessment tools

• Coordination of information about: – CBME initiatives in other ObGyn programs – Changes in residency program goals and

objectives – How CBME will affect our:

• clinical practice • teaching responsibilities and roles

• Information about faculty roles in the implementation of CBME

Departmental Challenges

• Improving faculty understanding of – CBME – CanMEDS 2015 – assessment tools and their use – feedback techniques

• Restructuring of clinics and ward to allow

for assessment and feedback

Change Management Schema

Incentives

Steps to Change

Incentives

Now it’s your turn

Each table will be given a departmental challenge

–Develop 1-2 ideas to address the challenge alone (5 min) and then as a table group (10 min)

–Prioritise the ideas (5 min)

–Present to the group as a whole (5 min each)

–Group discussion to prioritise as a whole (10 min)