Teaching Medical Students to Reflect “Deeper” Ilho Kang MD, Amy Hayton M.D, Raymond Wong MD, and Lawrence Loo MD Department of Medicine, Loma Linda University School of Medicine, Loma Linda, CA
Discussion
Conclusions
References
Introduction
While many studies have examined the importance of reflective writing in
medical education[1,2], there is a paucity of research evidence for any particular
intervention to improve the quality or “depth” of reflection among medical
students. To facilitate the development of critical reflection, a new curriculum
enhancement was introduced during the third year of medical school.
1. Plack MM, Greenberg L. The reflective practitioner: reaching for excellence in
practice. Pediatrics. 2005 Dec;116(6):1546-52
2. Stern DT, Papadakis M. The developing physician--becoming a professional. N
Engl J Med. 2006 Oct 26;355(17):1794-9.
3. Aronson L. Twelve tips for teaching reflection at all levels of medical education.
Med Teach. 2011;33(3):200-5
4. Mann K, Gordon J, MacLeod A. Reflection and reflective practice in health
preofessions education: a systemic review. Adv Health Sci Educ 2009;14:
695-621.
5. Wald HS, Borkan JM, Taylor JS, et al.: Fostering and evaluating reflective
capacity in medical education: developing the REFLECT rubric for assessing
reflective writing. Acad Med 2012;87:41-50.
6. Landis, J.R., Koch, G.G. (1977). The measurement of observer agreement for
categorical data. Biometrics. 33, 159–174.
In the initial analysis, our 90 minute educational intervention
successfully improved the depth of reflection by increasing the number
of students who wrote level four “critical reflection” papers. We believe
this short curriculum enhancement could readily be adapted by others
seeking to enhance student reflection and foster lifelong learning.
To our knowledge this is the first study to demonstrate that an
educational intervention to teach critical reflection improves the depth of
reflection by medical students.
•Strengths: The number of faculty evaluators and their successful
blinding that minimized bias in grading. The percentage agreement in
distinguishing “old” vs. “new” is similar to what one might expect by
chance (i.e. 50%) suggesting the authors were not able to distinguish
“old” versus “new” papers.
•Limitations: Low inter-rater reliability in using the REFLECT rubric.
Kappa statistic was used in our analysis and our value of 0.37 indicates
only “fair agreement”. This may limit the applicability of our results to
other institutions.
•Further Study: Development of a rubric that is more easily adaptable
and reliable may be valuable in future studies on reflection. We are
exploring the impact of an open-ended versus more directed prompt
on the depth of critical reflection. We are also considering whether
reflection level of students changes over the course of the academic
year.
In June 2012, a 90-minute interactive teaching session[3,4] was introduced to
all junior medical students rotating through the Internal Medicine clerkship at
Loma Linda University. Key components included:
Methods
The primary analysis showed that the number of students writing “critical
reflection” papers increased after the educational intervention from 19% (16/83)
to 43% (39/90) p = 0.0002 (Figure 1), using the Mann-Whitney U test. Inter-rater
reliability among the four faculty graders demonstrated only “fair” agreement, as
measured by a kappa statistic = 0.37[6].
Figure 1. “Critical reflection” pre vs. post new curriculum
Results
Table 1. REFLECT RUBRIC[5] Criteria used to judge reflection papers
Reflection Level Non-Reflective
1
Thoughtful Action
2
Reflection
3
Critical Reflection
4
Writing
Spectrum
Superficial descriptive
writing approach (fact
reporting, vague
impressions)
Elaborated descriptive
writing approach and
impressions without
reflection
Attempting to
understand, question,
or analyze the event
Exploration and critique
of assumptions, values,
beliefs, and/or biases
and the consequences
of action
Sense of Writer
Presence Writer partially present Writer partially present
Writer largely or fully
present
Writer largely or fully
present
Description of Conflict,
Disorienting Dilemma,
Challenge or Issue of
Concern
No description Absent or weak
description Description
Full description that
includes multiple
perspectives, exploring
alternative explanations,
and challenging
assumptions
Attending to
Emotions
Little or no
recognition or
attention to
Recognition but no
exploration or
attention to
Recognition,
exploration, and
attention to
Recognition, exploration,
attention to and gain of
emotional insight
Analysis & Meaning
Making None Little or unclear Some Comprehensive
Reflection level Non-reflective
1
Thoughtful
Action
2
Reflection
3
Critical
Reflection
4
Summary
Pre-
Intervention Number (%)
2
(2.4%)
30
(36.1%)
35
(42.6%)
16
(19.3%)
83
(100%)
Post-
Intervention Number (%)
2
(2.2%)
12
(13.3%)
37
(41.1%)
39
(43.3%)
90
(100%)
Column Total Number (%)
4
(2.3%)
42
(24.3%)
72
(41.6%)
55
(31.8%)
173
(100%)
Table 2. Summary of total written reflection papers (N=173)
The grades for the reflection papers prior to and after the introduction of the
curriculum enhancement are summarized in Table 2 & Figure 1.
The four faculty were asked to judged whether each reflection paper was “old”
(written prior to educational intervention) or “new” (after the intervention). The
proportion agreement = 44% and p > 0.05. The poor kappa statistic of 0.18
among the 4 faculty graders also suggested adequate blinding of this process.
Eighty three reflection papers written prior to the intervention were compared to
ninty reflection papers after the intervention. To minimize bias and blind the four
faculty graders, names and dates were removed from each typed reflection
paper. Differences in grading were discussed and resolved by consensus.
Four minute
video from the popular TV show
Scrubs, highlighting a
young doctor’s reflection
experience of a patient facing
death.
Four minute
video from the popular TV show
Scrubs, highlighting a
young doctor’s reflection
experience of a patient facing
death.
Large group discussion of the key concepts
and core components of
critical reflection.
Large group discussion of the key concepts
and core components of
critical reflection.
Small group interactive
exercise where students
compared and contrasted three
essays portraying different levels of
reflection.
Small group interactive
exercise where students
compared and contrasted three
essays portraying different levels of
reflection.
A faculty presenter shared
a personal reflection
critical to her own professional
development.
A faculty presenter shared
a personal reflection
critical to her own professional
development.
Students were then asked to write two reflection papers
during the 10-week rotation. A previously developed and
validated tool, the REFLECT rubric (Table 1)[5], was used to
grade the written reflection papers on four levels.
p = 0.0002
N
u
m
b
e
r
Level of Reflection5
Source: https://www.mededportal.org/icollaborative/resource/352
