competency based medical education and assessment maria lucarelli, md associate program director...

Competency Based Medical Education and Assessment

Maria Lucarelli, MDAssociate Program Director Internal Medicine Residency

January 13, 2015

2

Conflict of Interest

None

3

Describe the characteristics and benefits of competency assessments in medical education

Define types of competency assessments that may be applicable to your medical education area

Identify existing competency assessment tool in your medical education area

Objectives

4

Competency Based Education

Competency-based education (CBE) is an approach to preparing physicians for practice that is fundamentally oriented to graduate outcome abilities and organized around competencies derived from analysis of societal and patient needs. It deemphasizes time-based training and promises greater accountability, flexibility, and learner centeredness

J. R. Frank et al. 2010; 32: 631–637 Medical Teacher

5

Competency Based Education

Outcomes Based Evaluation integrates knowledge, skills and attitudes Time-independent Learning is Individualized Learning occurs in the workplace

Process versus Competency based Programs

6

Carraccio, C, Wolfsthal, SD, Englander, R, Ferentz, K, Martin C “Shifting Paradigms: From Flexner to Competencies” Acad Med (2002) 77:361-367.

7

Competency Based Education

Requires continuous, comprehensive assessment Majority of assessment should happen in clinical

environment Requires ongoing, highly effective feedback

8

Milestones

A significant point in development and helps define the appropriate trajectory of a trainee

Identify discrete knowledge, skills and attitudes expected of learners as they progress through training

9

Milestones

Can demonstrate individual trajectory of competency acquisition

Provides clear path of progress Can help focus assessment Allows for richer feedback

11

Competency

Observable Measurable Integrating multiple components such as knowledge,

skills, values, and attitudes Can be assembled for progressive development Descriptors of physicians

Competence

Abilities across multiple domains or aspects of performance in a certain context

Multi-dimensional and dynamic. It changes with time, experience, and setting.

Requires qualifiers Relevant abilities Context Stage of training

12

13

Competent

Possessing the required abilities in all domains in a certain context at a defined stage of medical education or practice.

Dreyfus Model

14

Expert

Proficient

Competent

Advanced Beginner

Novice

Statement of Awarded Responsibility

15

Expert

Proficient

Competent

Advanced Beginner

Novice

Providing supervision to others

Acting unsupervised

Acting with supervision available within minutes

Acting with direct

supervision present in the room

Observing the Activity

A case for competency-based anaesthesiology training with entrustable professional activities: An agenda for development and research.Jonker, Gersten; Hoff, Reinier; ten Cate, Olle

European Journal of Anaesthesiology. 32(2):71-76, February 2015.DOI: 10.1097/EJA.0000000000000109

16

17

Entrustable Professional Activities (EPA)

Specific knowledge, skills and attitudes acquired over the course of training critical to performing as a physician

Ten Cate Academic Medicine, Vol. 82, No. 6 / June 2007

18

EPA

Competency

Subcompetencies

Milestones

19

 End-of-

Training EPA 

Step 1Description and Tasks

Step 2Related Curricular Milestones

(Abbreviations on AAIM Website)

Step 3Assessment

Methods/Tools

1. Manage the care of patients with acute common diseases across multiple care settings

Internal medicine physicians entering into unsupervised practice are able to diagnose and manage common acute medical symptoms (e.g., joint pain, chest pain, and headache) and conditions (e.g., uncontrolled HTN, decompensated HF, and COPD exacerbation) in community, ambulatory, and hospital settings.  The tasks required: Obtain accurate and complete

information sufficient to develop differential diagnosis and inform care plan;

Knowledge of diseases common to internal medicine;

Communicate plans of care to patients, families and care givers

Adapt care plans to changing clinical information

Patient Care (PC) A2, A3, B1, B2, C2, C3, D1, E1, F8

Multisource feedbackChart stimulated recallChart auditsDirect observationsStandardized patient/OSCEIn-training examination

Medical Knowledge (MK) A2, A3, B1, B3

Practice-Based Learning & Improvement (PBLI)

B1, B2, B3, D4, E1, E2

Interpersonal & Communication Skills (ICS)

A3, A4, A5, A7, B1, B3, D3, F1

Professionalism (P) B1, B3, D2, E1, F1, F2, F3 G2, H1, I1, I2, K3

Systems-Based Practice (SBP)

A3, B2, D2, D4, E1, E3

Building Assessments for an EPA in Three Steps. AAIM Conect

© 2015 European Society of Anaesthesiology. Published by Lippincott Williams & Wilkins. 2

Fig. 1A case for competency-based anaesthesiology training with entrustable professional activities: An agenda for development and research.Jonker, Gersten; Hoff, Reinier; ten Cate, Olle

European Journal of Anaesthesiology. 32(2):71-76, February 2015.DOI: 10.1097/EJA.0000000000000109

Fig. 1 . Acquisition of competence. (a) Acquisition of competence, showing the competence threshold (corresponding to proficiency level IV) and continuing growth of expertise after delegation of a clinical activity. (b) Acquisition of competence for 5 different EPAs. At reaching the competence threshold, an informed and justified delegation decision can be made. Adapted with permission from 20.

Role of Assessment

21

Deficiencie

s

Advanced

• Earlier identification• Customized

remediation plans• Earlier and fair exit

from training when necessary

• Earlier identification• Customized training• More rapid

advancement

Holmboe et al The role of assessment in competency based medical education. 2010; 32:676-82. Medical Teacher

Assessments

Needs to be continuous and frequent Formative > summative Allows for deliberate practice

Needs to be coupled with effective feedback and ongoing mentoring/coaching

22

Assessments

Criterion Based Allows for appropriate expectations Developmental (milestones) Blueprint for assessment and informs appropriate

methods/tools for assessment

Take place in the clinical environment Does not eliminate the role of simulation Heavily dependent upon faculty

observations/assessment

23

24

Assessment

Requires the use of high quality assessment tools One single evaluation may not be perfect Consider the use of multiple tools

Qualitative approaches to assessment can be valuable

25

Narrative descriptions should replace grades and numerical ratings for clinical performance in medical education in the United StatesJanice L. Hanson*, Adam A. Rosenberg and J. Lindsey Lane Front. Psychol., 21 November 2013

26

360 evaluations

Patient surveys

Knowledge assessments

Mapping Milestones

27

A Tool for Mapping the ACGME Milestones to Evaluation SourcesLinda Myerholtz Ph.D.

28

Resources

PM&R Milestones Central www.physiatry.org

Society of Teachers of Family Medicine www.fmdrl.org

Alliance For Academic Internal Medicine www.im.org