✶  Why – clear need to be met (learner/patient) ✶  What – specific outcome objectives

focused on the need ✶  When, where, how - targeted intervention

strong enough to make a difference ✶  Evaluation - laser focused on targeted

objectives

✶  The idea - one sentence – 25 words

✶  Need/rationale - one paragraph -context

and need – 125 words

✶  Method - one paragraph – 225 words ✦  Project outcome objective

✦  Innovative method - appropriate to meet need

✶  Evaluation – 100 words

✶  Impact on Profession – 25 words

Criteria

Items Rating Comments to

authors 1. How clear was the “cool idea?” 3 – Very clear, well described 2 – Okay, could understand it 1 – Not clear, hard to figure out what the idea is

2. How well was the need/problem identified in the abstract? 3 – Very clear, well described 2 – Okay, could tease it out 1 – Not defined

3. How well were the proposed methods and evaluation described?

3 – Very clear, well described 2 – Good start, still needs work 1 – Not included in abstract

4. How great an impact do you think this project might have? 3 - Great potential, good transferability, exciting 2 - Some potential, local impact or single specialty 1 - Cannot tell or small impact

5. Mark One Below: Reject (mostly 1’s) ___ Accept as Poster (narrow interest or mostly 2s) ___ Accept as Presentation (broad interest and mostly 3s) ___

Comment for IME Program Committee only

! Stated in one sentence - 25 words - three

elements

! The target audience (patient, learners,

providers, environment)

! What you want to be “better” at the end of

your intervention or training (outcome

objective)

!  Your method

The idea - one sentence

Example - Use the Nintendo Wii as a medical

simulator to build procedural skills in first and

second year medical students

Key Elements

1.  Technique/innovation - HOW

2.  To accomplish xxx - outcome

objective - WHAT

3.  Target group - WHO

The idea - one sentence Example - Improve situational awareness and self-modifying behaviors of family medicine residents through monthly guided-discussions using the TEM (threat and error management) format to analyze resident errors.

Key Elements

1.  Technique/innovation - HOW

2.  To accomplish xxx - WHAT

3.  Target group - WHO

" Take out a piece of paper

" Write Who, What and How (spaced widely down the page)

" Who – write your target audience – be sure to specify if it is a single group within residency – also state the total number of learners (e.g. 6 PGY2 FM residents)

" What – state the focus of your cool idea

" A. State the target competency area

" For patients – medical problem, aspect of care, etc.

" For learners – MK, PC, ICS, Prof/CC, PBLI, SBP

" B. Be specific as possible

" How – to do “how” must know first do when and where

(1) Where - Write down where – e.g. outpatient clinic, core curriculum, special workshop or retreat, inpatient rounds, etc.

(2) When – Write down how much total time you have been allotted (or might realistically be given) for your “intervention” – e.g. 3 months; one rotation; 30 minutes each month as part of core curriculum; 5 minutes at the beginning and 5 minutes at the end of each clinic session;

" How – now that you have your when and where completed you can think methods – pick one or more and write one sentence about how you might utilize it in your setting

The idea - one sentence Example - Improve situational awareness and self-modifying behaviors of family medicine residents through monthly guided-discussions using the TEM (threat and error management) format to analyze resident errors.

Key Elements

1.  Technique/innovation - HOW

2.  To accomplish xxx - WHAT

3.  Target group - WHO

"  Need based on:

"  Gap between current and optimal

"  Perceived need

"  Rationale for proposed method

"  Can include both need for intervention and rationale for method selected

"  Brief - no more than 3 references – can number them, 1,2,3 (submission form allows for up to 3) – 125-150 words

Rationale: There are many mistakes made in outpatient settings. One 2004 study of practicing physicians reported that almost 25% of visits had some form of mistake, with up to 25% of those resulting in harm (1). In our practice, faculty members have observed that residents are often unable to identify their own mistakes, although they exhibit the skill to identify mistakes made by others. This is an issue faced by all training programs that have an outpatient component and by all physicians no matter where they will ultimately work. We believe that it is possible to help learners realize that errors are common but can be prevented. Based on work done by others (2), we have planned an intervention aimed to make discussion of errors an ongoing activity and to improve attention on practices that can enhance safety and improve comfort with self-reporting of errors of omission or commission. 1. Elder NC; Meulen MV. Cassedy A. The Identifi cation of Medical Errors by Family Physicians During Outpatient Visits. Ann Fam Med 2004;2:125-129. 2, Neuspiel DR; Stubbs EH; Liggin L. Improving reporting of outpatient pediatric medical errors. Pediatrics. 128(6):e1608-13, 2011 Dec.

Method - one paragraph including ✦  One sentence describe target audience,

length of intervention and site for intervention

✦  One sentence of key outcome objective ✦  Describe innovative method -

appropriate to meet need ✦  List key steps for implementation ✦  May include comparison group (turning

intervention into research study)

Methods/Description: PGY-2 anesthesia residents (N=10) will attend 8 monthly SBP 2-hour sessions facilitated by faculty trained in lean sigma. Learners will analyze real-life cases, view videos, and participate in games from The Systems Thinking Playbook. Individual “All Systems Go” workbooks will designate SBP-related tasks that residents will perform with faculty during clinical rotations. Finally, faculty-mentored resident teams will identify a quality issue and complete a systems analysis worksheet that outlines their strategic plan for improvement. Teams will present project proposals at Grand Rounds for faculty input, revision and implementation during the following year. At the end of the year, learners should be able to: 1) describe their role within systems 2) discuss key systems- thinking concepts, 3) apply systems-thinking principles to effect change and 4) express confidence and willingness to improve healthcare systems. At the end of the year, residents will complete a survey on SBP knowledge/attitudes. PGY-2 anesthesia residents in another California anesthesiology programs will serve as controls.

1 Accountability – did what was promised?

2  Reaction – did they like it?

3  Learning – what did they learn (knowledge,

skills, attitudes)?

4  Behavior – what are they doing differently?

1.  Numbers of session or programs given

2.  Numbers of attendees at various programs – use

attendance forms

a. total attendees (patterns)

b. unique attendees (data about)

3.  Numbers of leaner projects completed

4.  Numbers of certificates awarded, etc

What? " Satisfaction " Opinion on quality

of teaching, sessions, materials

" What they report learning

" Commitments to act/change behavior

How? " Rating forms " Focus groups

" Reflections

How? - Guidelines " Use control group if

practical

" Laser focus on key objectives

" Measure before and after (if no control)

" Get 100% response

Tools? " Examination

#  Oral, written, digital

" Observation #  Performance

" Documentation #  Portfolio

#  Attitude survey

Why Assess? " Application of new

K, Sk, Att is a goal

Guidelines " Use control group

" Use time series " 100% response rate

How? " Surveys completed

by teachers/bosses

" Observation " Review of products

" Review of artifacts " Interviews/focus

groups to follow-up commitments to act

How will you know it worked

✶  “Up-line” assessment - need, obj, plan,

context, implementation/process

✶  Learner reaction (Level 1)

✶  Outcome - impact on:

✶  Leaner performance (K/Att/Sk)

✶  Learner practice behaviors

✶  Impact on patients, institution, etc.

✶  Expert review of plan or implementation

✶  Tracking: activities completed, attendance, etc.

✶  Learner opinion survey (reaction)

✶  Outcomes

✶  Exams, Surveys (K/Att)

✶  RS for Direct Obs, MSCE (Sk)

✶  Patient surveys, chart review (Beh/Impact)

✶  Commitment to Change with follow-up (Beh)

Evaluation: Learner reaction will be assessed through a program evaluation at the end of the PGY-2 year. A post-survey of faculty comfort with SBP assessment will be compared with their pre-curriculum survey results. Learning will be evaluated by a pre and posttest on course content and project proposals scored by adherence to systems thinking principles. Learner behavior will be assessed through a end of course retrospective pre-post survey of confidence and behaviors. Resident compliance data with systems protocols will be compared against historical data. Resident test and survey data will be compared to control group data.

Impact on Field: If we want future physicians to advocate for sustained solutions in healthcare inequity, cost and safety, then we must equip residents with cognitive and behavioral tools that allow them to understand and access complex systems. This SBP curriculum is designed to do so and if successful could be a model for programs nationally.

✶  The idea - one sentence – 25 words

✶  Need/rationale - one paragraph -context

and need – 125 words

✶  Method - one paragraph – 225 words ✦  Who, when, where, key outcome objective

✦  Innovative method - appropriate to meet need

✶  Evaluation – 100 words

✶  Impact on Profession – 25 words