University of Rochester Medical Center
Structuring M & M Conferences for Educational Effectiveness
M.J. Kim, F.J. Fleming, J.H. Peters, R.S. Salloum, J.R.T. Monson, M. Eghbali Department of Surgery University of Rochester, Rochester, NY April 22, 2010
University of Rochester Medical Center 2
Background
- Morbidity & Mortality Conferences:• Required by ACGME for surgical residents
• “Practice Based Learning and Improvement”
1. Analyzing patient complications
2. Identifying causes
3. Proposing potential solutions
University of Rochester Medical Center 3
Background
- Content/format discrepancies Aboumatar et al, American Journal of Medical
Quality 2007
- Mixed lessons learnedBender et al, American Journal of Surgery 2009
- Structuring M&M presentations can improve• Understanding of complications
• Learner satisfactionRisucci et al, Current Surgery 2003
Muryama et al, American Journal of Surgery 2002
University of Rochester Medical Center 4
Objectives
1) Evaluate effectiveness of existing M&M conferences
• Content/Delivery
• Resident learning
• Perceptions of effectiveness
2) Develop a standardized presentation format
•Missing content
• Streamline discussion
3) Determine effectiveness of intervention
University of Rochester Medical Center 5
Definitions
Educational Effectiveness
- Specificity of the cause of the complication
- “OR staff not notified that potentially unstable
patient in ED” vs. “Patient had delay to OR”
- Specificity of the potential solution/change
- “Designate contact person in OR for
potential emergency cases” vs. “Need
better communication”
University of Rochester Medical Center 6
Methods
Study Design
•Pre/Post intervention evaluation
Study Sample
•Conferences in five divisions
•All presentations by residents
•Audience of faculty, residents, students,
and staff
University of Rochester Medical Center 7
Methods
- Conference Observation (TEACHING)•Two independent observers
•Consensus regarding conference practices
- Resident Questionnaires (LEARNING)•2 Questions + any comments
- Surveys of Learners (PERCEPTIONS)•Online, anonymous
•Last M&M conference attended
Results: Observation
Observation Item %Yes (n = 40)
1) Is the complication clearly stated? 80%
2) If so, is it stated by the resident
presenter?55%
3) Is a potential cause clearly
established?65%
4) Is the cause proposed by the
presenter?18%
5) Are strategies for practice change
discussed during each case?85%
6) Are the changes proposed by the
presenter?8%
University of Rochester Medical Center 9
Results: Questionnaires
Specificity of cause of complication (n=26)•Mean rating (±SD) = 3.11 (±1.48)
• 1 = Vague
• 5 = Specific
Specificity of a practice change (n=26) •Mean rating = 3.42 (±1.50)
• 1 = No practice change stated
• 3 = Vague practice change
• 5 = Specific practice change based on current discussion
Results: Survey
Question %Yes (n = 19)
1) Is the current format of M&M
presentations effective for analyzing
complications?
63%
2) Is the current format effective for
learning?53%
3) Are specific complications clearly
stated?47%
4) Are specific causes clearly stated? 21%
5) After presentation, is it clear how to
avoid this complication in the future?32%
University of Rochester Medical Center 11
Presentation Format
Case Presentation (< 5 minutes)• Introduction•Clinical Information•Analysis
Supporting Information (< 5 mins)•Literature/Background Review•Take Home Points (Maximum 2!)
Discussion (< 5 mins)
= Total Time < 15 minutes per case
University of Rochester Medical Center 12
Presentation Format
Format based on•Suggestions from surveys•Previous publications
Introduced to residents•Sample presentation demonstrated• Template slides with instructions
Introduced to faculty•Format/process discussed at faculty meeting
Template, sample, instructions sent to all residents, fellows, and faculty
University of Rochester Medical Center 13
Results: Observation
* p<0.01
Observation Item Pre-Format
%Yes (n = 40)
Post-Format, %Yes (n = 35)
1) Is the complication clearly stated? 80% 100%*
2) If so, is it stated by the resident
presenter?
55% 100%*
3) Is a potential cause clearly established? 65% 91%*
4) Is the cause proposed by the presenter? 18% 57%*
5) Are strategies for practice change
discussed during each case?
85% 89%
6) Are the changes proposed by the
presenter?
8% 54%*
University of Rochester Medical Center 14
Results: Questionnaires
Pre-Format
Mean (±SD)
Post-Format
Mean (±SD)
Specificity of the
cause of complication 3.11 (±1.48) 4.56 (±1.03)*
Specificity of a future
practice change 3.42 (±1.50) 4.31 (±1.40)*
*p<0.05
University of Rochester Medical Center 15
Results: Survey
Question Pre-Format,
%Yes (n = 19)
Post-Format,
%Yes (n = 18)
1) Is the current format of M&M
presentations effective for analyzing
complications?
63% 89%
2) Is the current format effective for
learning?
53% 89%
3) Are specific complications clearly
stated?
47% 89%
4) Are specific causes clearly stated? 21% 72%
5) After presentation, is it clear how to
avoid this complication in the future?
32% 78%
University of Rochester Medical Center 16
Conclusions
1) Structuring presentations leads to more
specific lessons taught during M&M
2) More specific lessons from resident
presenters leads to better learning by peers
3) Structuring presentations is a practical
intervention to improve M&M conferences
University of Rochester Medical Center 17
University of Rochester Medical Center 18
University of Rochester Medical Center 19
University of Rochester Medical Center 20
Results: Survey
Suggested areas for changes:
•Focused discussion period - 79%
•Established presentation format - 63%
•Time-limited presentations - 53%
•Resident-led discussion - 21%
•Other (specify improvements, streamline) - 21%
University of Rochester Medical Center 21
Results: Observation
* p<0.01
Observation Item Pre-Format,
%Yes (n = 40)
Post-Format,
%Yes (n = 35)
1) Is the complication clearly stated? 80% 100%*
1b) Within 5 minutes? 20% 100%*
2) If so, is it stated by the resident presenter? 55% 100%*
3) Is a potential cause clearly established? 65% 91%*
3b) Within 10 minutes? 10% 71%*
4) Is the cause proposed by the presenter? 18% 57%*
5) Are strategies for practice change discussed
during each case?
85% 89%
6) Are the changes proposed by the presenter? 8% 54%*
University of Rochester Medical Center 22
Results: Extent of use
Did residents use the new format?• Mean = 3.57 ±1.40 (1 = No organization,
5 = Followed each step)
Was there literature review?• Mean = 2.71 ±1.71 (1 = No attempt at supporting info,
5 = clearly applicable studies cited)
Did faculty wait to ask questions?• Yes = 17%
Was each case <15 minutes?• Yes = 51%
University of Rochester Medical Center 23
Future Directions
1) Evaluate long-term (6-8 months)
adherence to format changes
2) Study when/why format is not used
3) Compare with other assessments to
evaluate extent of benefit