implementation and experience with a standardized

Implementation and Experience with a Standardized Assessment of a

Clinical Encounter Report (STACER) in a Surgical Residency Program

Morgan Langille MD, Erin Wright MDCM, MEd

Division of Otolaryngology-Head & Neck Surgery University of Alberta

Disclosure

• EDW • Speaker – Schering/Merck, GlaxoSmithKline • Travel Grant – Medtronic • Consultant – J & J Medical Products

Introduction

• In – Training Evaluation Reports (ITER)

– End of rotation evaluation – Based on overall impression of resident’s progress

on a given rotation • Generally a summative assessment

Introduction

• In – Training Evaluation Reports (ITER) – Ideally based on objective (and preferentially

DOCUMENTED) assessments • Comprehensive • Document direct observational evaluations

– Clinic – Surgical setting

Evaluation – General Concepts

• Summative evaluation: – Can the resident perform the surgery successfully? – Typically quantitative

• Formative evaluation: – Evaluations intended as the basis for improvement – Offers residents feedback

Evaluation: Formative vs. Summative

"When the cook tastes the soup, that’s

formative; when the guests taste the soup, that’s summative."

- Robert Stakes

STACER

• Standardized Assessment of a Clinical Encounter Report – Initial version designed by Specialty Committee

(Otolaryngology-Head & Neck Surgery)

• Intent: – Enhance formative aspect of evaluation – Robust assessment of performance

• Direct and objective assessment • Immediate feedback

Surgical STACER items

1. Pre-operative plan 2. Knowledge of indications & contraindications 3. Obtains an informed surgical consent 4. Communication, plan of surgery, including

safe surgery checklist 5. Pre-incision planning, including equipment

preparation, imaging, patient positioning

Surgical STACER items

6. Incision, flap elevation, exposure and ID of landmarks

7. Dexterity, hemostasis, surgical technique 8. Decision making related to inter-operative or

unexpected findings 9. Wound closure and dressing 10.Post-operative orders and communication of

care plan 11.Documentation of peri-operative encounter

Clinical STACER

1. History of presenting complaint 2. Co-morbid issues related to present

complaint 3. Past medical history, medication history 4. Social history 5. Organization of interview

Clinical STACER

6. Physical examination 7. Case Presentation, including organization,

succinctness 8. Investigations, including interpretation of

past investigations 9. Diagnosis/Differential Diagnosis 10.Care Plan and Patient Counseling/Education 11.Surgical Consent (if applicable)

Implementation

• Staff and Residents oriented to STACER during

divisional grand rounds

• Residents asked to complete 2 STACERs per rotation: – 1 Clinical – 1 Surgical

Implementation

• STACERs collected over 1 year

• Added to residents’ ongoing portfolio

Feedback on STACER Program

• A Semi – Structured Questionnaire was

designed for STACER feedback

• Responses received from 100% residents and 60% of staff

Semi – Structured Questionnaire

• What was your initial opinion/expectation when the STACER evaluation tool was implemented? – Skeptical – More paperwork – Time consuming – “Not another evaluation!” – Good idea

Semi – Structured Questionnaire

• What were your impressions with the implementation process? – Easy – Multiple reminders helpful

• Program Director • Designated Resident

– Circulated in advance – Hard copies available in multiple locations – Several aspects were “not applicable”

• Variation in case selection

Semi – Structured Questionnaire

• What was your initial experience with the STACER tool? – Somewhat unfamiliar at first – Confusion around logistics short lived – Many experienced no problems – Good “Snapshot” of performance

Semi – Structured Questionnaire

• What is your current perspective of the STACER tool? – Provides useful feedback – Surgical STACER particularly relevant – Turns a typical case into a learning experience – Encourages honest assessment

Semi – Structured Questionnaire

• What are the benefits/value of STACERs? (Has it made evaluation better and how?) – Learn tips and tricks that would otherwise be

missed – Residents use the STACER to “claim” a case – Facilitates discussion – Practical

Semi – Structured Questionnaire

• What are the perceived downsides to STACERs? Challenges to their use? – Remembering to complete prior to end of rotation – Awkward having Staff watch clinical STACER – Only provide a single snapshot

Semi – Structured Questionnaire

• What CanMEDS roles do you find best assessed by the STACER tool? – Manager – Health Advocate – Professional – Communicator – Collaborator

STACER - Conclusions

• Benefits: – Enhances the validity of our ITERS

• Specific documentation and evidence

– Royal College reviewer liked our STACERs – Applicable to any surgery / Clinic – Standardized – Well received by our division

STACER - Conclusions

• Drawbacks – Less objective outcome measures * – Lack of validation for specific procedures (but is

applicable to almost any procedure)

Questions?

• Thank – You • morgan.langille@gmail.com