the center for medical simulation
TRANSCRIPT
1
The Center for Medical Simulation
DEBRIEFING ASSESSMENT
FOR SIMULATION IN HEALTHCARE (DASH)©
Rater’s Handbook
2
INTRODUCTION
A crucial step in clarifying and consolidating the learning gained fro
m simulations in healthcare is to debrief those simulation experien
ces—
that is, to methodically review what happened and why. Leading a
debriefing is a learned skill, and can be improved with practice and f
eedback. The Debriefing Assessment for Simulation in Healthcare (
DASH)© Handbook and Rating Forms are designed to assist in evalu
ating and developing debriefing skills. Additional information, rating forms, and contact information can
be found at the DASH website: http://www.harvardmedsim.org/da
sh.html.
Background on the Debriefing Process In the context of healthcare simulation, a debriefing is a conver
sation among two or more people to review a simulated event o
r activity. In the debriefing, participants explore and analyze thei
r actions and thought processes, emotional states,
and other information to improve performance in real situations. When instructors foster high participant engagement this yields be
tter retention and deeper learning, and increases the likelihood of t
he transfer of new or reinforced knowledge, skills, and attitudes to
the clinical or broader healthcare setting.
Debriefers make an implicit comparison between a desired level of
performance and the level of performance they observe in the sim
ulation. The difference between the desired performance and the
actual performance is called the performance gap. The gap can be
large, small, or, in the event of good or excellent performance, zer
o. Describing their assessment of this gap and inquiring into its ori
gins is part of an effective debriefing.
Debriefers can help improve poor performance or reinforce good
performance by critiquing and discussing trainees’ actions. Alterna
tively or additionally, they can explore the “frames”─ internal und
erstanding and assumptions about external reality─that drove trai
nees’ actions. Almost all complex actions are driven by frames. De
briefings can focus on frames or actions or both.
Healthcare professionals take psychological risks when they allow
their performance to be watched and analyzed by peers and instru
ctors. Thus, developing and maintaining a psychologically safe lear
ning environment is essential for getting everyone’s wholehearted
participation. To do this, skilled debriefers give participants the be
nefit of the doubt (i.e., assume the best intentions) and regard mis
takes and good performance as a mystery to be analyzed rather th
an as a crime to be punished or success to be simply lauded.
Feedback on current performance is a crucial part of the
debriefing process. Skilled debriefers are neither overly harsh and
negative, nor falsely kind and non-
‐judgmental. Rather they provide direct, respectful feedback and ex
plore the underlying drivers of a trainee’s performance. Most debri
efings aim to improve deficits in performance; but debriefings can a
lso explore good performance: What actions or thought processes
allowed a person or team to be effective?
What Is the DASH?
The DASH assesses the instructor behaviors that facilitate learning
and change in experiential contexts. The behaviors described in th
e DASH were derived from relevant theory and evidence in previous
research as well as expert observations of hundreds of debriefings.
The DASH is designed to guide assessment of debriefings in a
variety of healthcare disciplines and courses, involving differing
3
numbers of participants and a wide range of educational objective
s, from an exercise in suturing skills to one in disaster management
for a whole hospital. It works in a variety of settings with various p
hysical and time constraints. The DASH can be used to assess the in
troduction to the simulation course (Element 1) and the post-
‐simulation debriefing (Elements 2-
‐6). How well the instructor sets up an environment conducive to l
earning significantly affects the subsequent activities; scoring the in
troduction will enrich the meaning of the overall score.
The DASH tracks and rates six key Elements of a debriefing. These include whether and how the instructor
(1)
Establishes an engaging learning environment; (2)
Maintains an engaging learning environment; (3)
Structures debriefing in an organized way;
(4) Provokes engaging discussions;
(5)
Identifies and explores performance gaps; and (6)
Helps trainees achieve or sustain good future
performance.
The DASH is a behaviorally anchored rating scale, which means it is
based on the behaviors needed to execute an effective debriefing
as well as those characteristic of poor debriefings. An "Element" in
the DASH is a high-
‐level concept that describes a whole area of debriefing behavior.
Each Element is composed of "Dimensions" that reflect parts of the
high-
‐level competency. The bulleted “Behaviors” within each Dimensio
n are concrete examples of carrying out this element. The listed be
haviors are examples; possible behaviors are not limited to those li
sted.
Directions for Scoring
Raters score each Element using the dimension descriptions and ex
ample behaviors as guides. A complete DASH score includes five (or
six, if Element 1, the introduction to the course, has been
4
observed) Element ratings. Ratings are based on a seven-‐point
effectiveness scale.
Rating Scale
Rating Descriptor
7 Extremely Effective / Outstanding
6 Consistently Effective / Very Good
5 Mostly Effective / Good
4 Somewhat Effective / Average
3 Mostly Ineffective / Poor
2 Consistently Ineffective / Very Poor
1 Extremely Ineffective / Detrimental
As a rater, you must study the Elements and be completely familiar
with them. Summarize your impression of overall effectiveness for
the whole
Element, guided by your observation of the individual dimensions
and behaviors that define it. Think holistically and not arithmeticall
y to consider the cumulative impact of the dimensions, which may
not all bear equal weight. You, the rater, will weigh dimensions as y
ou see fit, based on your holistic view of the Element. If a defined
dimension is not impossible or not available to assess, don’t let tha
t negatively influence your evaluation. For example, if no participa
nts got upset, just disregard the dimension regarding how the debri
efer handles upset people.
Once raters are thoroughly familiar with the Elements, they re
port that the scale is easy to use. Trained raters are able to pro
duce reliable and valid scores.
It is expected that short debriefings and ones that focus only on
actions and do not address frames are unlikely to be scored
5
higher than a 4, 5, or occasionally 6. It should be understood that t
he setting of the debriefing might affect the ratings. If the setting f
or the debriefing is less than ideal—
a cramped room close to real world distractions, or in the same spa
ce as the simulation—
with only a few minutes to spare, then it is reasonable to see the be
st ratings reach only “somewhat or mostly effective.” When particip
ants feel time pressure, lack privacy, or are too close to
real clinical or managerial distractions, it is difficult to pursue an i
n-
‐depth analysis of the thinking behind behaviors (the causes or “fr
ames” for actions).
There is purposeful
overlap between the Elements. For example, Element 1, “Establis
hes an engaging context for learning,” has
some of the same Dimensions as Element 2 “Maintains an engaging
context for learning.” In all cases and for every Element, raters sho
uld ignore the overlap and rate each Element
independent of the others.
Score Sheets Two different score sheets are available for raters. The short form
, designed to obtain Element scores only, is appropriate especially f
or summative evaluations. The long form, designed to obtain Dim
ension and Element scores, is especially useful when providing for
mative feedback to debriefers.
© Center for Medical Simulation, 2010
6
ELEMENT 1 Establishes an engaging learning environment
How well the debriefer or instructor introduces the simulation
learning experience can set the tone for all that follows. Before
any simulation begins, the instructor helps participants be clear
about what is expected of them, and helps them understand the
benefits and limits of the simulated clinical setting. The instructor
informs trainees whether and how the case, event, or procedure
will later be debriefed (i.e., discussed and analyzed) and whether
the simulation will be videotaped. The instructor makes plain that
the focus is on learning, not on “catching” people in a mistake,
and creates an environment where participants feel safe in
sharing thoughts and feelings about the upcoming simulation and
debriefing without fear of being shamed or humiliated.
Element 1 Dimensions
Clarifies course objectives, environment, roles, and expectations.
Simulation-
‐based courses flow better and participants engage more when
they understand the point of what they are doing, their role, the
instructors’ roles, and what is expected of them.
Positive, effective behaviors for this dimension include:
• Introducing oneself and inviting others to introduce themselves.
• Sharing and inviting others to share information about their pers
onal qualifications and experience, background, and interest in th
e course.
• Presenting the course overview and learning objectives.
• Introducing and describing the simulators, ancillary equipment,
location of supplies, role of actors, etc.
• Modeling expectations for participants to engage with equipment
(e.g., staying “in genuine and earnest character” when taking care
of a simulated patient).
• Discussing the origins of the cases, procedures, or events to be
simulated and why certain cases or tasks have been included in th
e course (e.g., actual cases that had bad outcomes; they are parti
cularly difficult or amenable to practice, etc.).
• Explicitly stating expectations for participants’ roles in the
simulation course and subsequent debriefing.
• Requesting that participants engage in debriefing discussions
and attempt to be self-‐reflective.
• Stating etiquette rules for debriefing: to be respectful, curious,
and polite about others’ thoughts and actions.
• Encouraging people to speak up, or allowing for respectful
disagreement.
• Explicitly describing the instructor’s role: to facilitate discussion; c
ommenting on performance based on observing similar events or si
mulations; acting as a resource on own area of expertise (e.g., PALS
, ACLS; CRM; teamwork; clinical); and ensuring that the training obj
ectives are met.
Negative, ineffective behaviors include:
• Not describing objectives, roles, or expectations.
• Being vague or misleading in describing the simulation or debriefing.
• Leaving no time or opportunity for student questions.
7
Establishes a “fiction contract” with participants.
The fiction contract is a joint endeavor that debriefers and studen
ts create. In it, the instructor acknowledges that the simulation ca
nnot be exactly like real life but agrees to make the simulation as
real as possible within resource and technology constraints. Partic
ipants agree to do their best to act as if everything were real.
Positive, effective behaviors for this dimension include:
• Explaining that the instructor and participants have to c
ollaborate to create an engaging simulation and learning e
nvironment.
• Stating that the instructor‘s obligation is to do everything to
make the simulation as real as possible within resource and tec
hnology constraints.
• Asking the participants to do their part to act, as best they can,
as if the simulation were real, acknowledging that a participant
will likely act differently in the simulation environment than in the
real clinical environment.
• Stating a fair and balanced assessment of simulator strengths
and weaknesses.
Negative or ineffective behaviors include:
• Trivializing the challenges students face in “buying in” to the
realism of the simulation.
• Stating or assuming that trainees should and will act the same
way they would in the real clinical setting.
• Insinuating or stating that it’s the student’s fault if the
simulation doesn’t seem real to them.
Attends to logistical details.
Although it may seem secondary, informing participants about l
ogistical details and providing a physically comfortable environ
ment helps them focus on learning.
Positive, effective behaviors for this dimension include:
• Making sure that the learning space or conference room is cle
an. When available, chairs, tables, white board, video, simulati
on devices, or other educational materials are orderly, clean, a
nd ready when the participants arrive.
• Briefing participants on where the simulation will take place and
how long it may last.
• Letting participants know about the availability of food and
drink, transportation or logistical considerations, locations of
bathrooms, etc.
• Informing participants about when and where the simulated
case, procedure, or event is likely to be debriefed.
Negative or ineffective behaviors include:
• Not orienting participants to course logistics and the physical
environment.
• Ignoring or making light of trainees’ concerns about timing,
location, or physical needs.
• Failing to address individuals’ potential challenges related to t
heir particular physical circumstances (e.g., do they use a wheel
chair or other device, do they have a latex allergy etc.)
Conveys a commitment to respecting learners and
understanding their perspective.
Participants often worry that simulations are designed to expose
their weaknesses or to humiliate them. To counter these notions,
8
instructors should offer a clear alternative approach by conveying t
hat they assume the trainee has good intentions and are trying to do
their best but will likely make mistakes along the way – which
is perfectly all right because this is a good place to talk about
improving our practice.
Positive or effective behaviors in this dimension include:
• Stating something like, “Mistakes are puzzles to be solved, not
crimes to be punished.”
• Stating that he or she understands that trainees are trying to
accomplish something positive, even when they make mistakes.
• Expressing some kind of “basic assumption” that trainees are i
ntelligent, well-
‐trained, and are trying to do their best to learn and improve.
Negative, ineffective behaviors include:
• Teasing, belittling, or ignoring participants’ expressions of
anxiety.
• Stating or implying that poor performance by trainees in the
simulation will be held against them.
• Making demeaning comments about a student’s competency.
• Saying things that undermine a student’s aspiration to be a
capable health care provider.
9
ELEMENT 2 Maintains an engaging learning environment
This Element clearly overlaps Element #1 and encompasses how
well the debriefer maintains a good learning environment. At the
beginning of the debriefing, especially if it is the first of the day,
and throughout the debriefing, the instructor should help trainees
stay clear about what is expected of them. The instructor should
ensure that students feel confident that the environment is safe
for sharing thoughts and feelings─ that is, that they won’t be
shamed or humiliated─ and that the focus is on learning, not on
“catching” people in a mistake.
Element 2 Dimensions
Clarifies debriefing objectives, roles, and expectations.
Debriefings flow better and participants engage more when they
understand the point of what they are doing, their role, the instru
ctors’ roles, and what is expected of them.
Positive, effective behaviors include:
• Stating topic areas that will be covered (e.g. “I’d like to discuss
how to identify arrhythmias.”)
• Stating expectations for participants’ role in debriefing.
• Requesting that participants engage in discussions and attempt
to be self-‐reflective.
• Explicitly describing the debriefer’s own role: to facilitate discuss
ion; to comment on performance based on having observed simila
r simulations; to act as a resource on own area of
expertise (e.g., PALS, ACLS, CRM, teamwork, clinical); to ensure
that the training objectives are met. Neg
ative, ineffective behaviors include:
• Saying nothing about the roles of debriefer or participants.
• Leaving debriefing objectives vague or unstated.
• Allowing no time for questions or comments from students.
• Not adjusting to verbal or non-‐verbal cues from participants
that indicate they are confused about goals.
Helps participants engage in a limited-realism context.
Participants often don’t appreciate (or resent) the constraints of si
mulation realism until they have been through a specific simulation
. They want to perform well and often feel badly when they don’t.
If they feel tricked or frustrated by the simulation technology or wo
rry they won’t be seen as competent, they will complain. Sometim
es they are simply frustrated when limited realism confounds their
problem-
‐solving process. Debriefers can help trainees address these issues
by acknowledging the legitimacy of their complaints, yet keeping t
he debriefing focused on the learning objectives, not the equipme
nt. Instructors model the seriousness and realism of scenarios or procedural simulations by treating them, when appropriate, as “cases” and treating mann
equins, standardized/actor patients, and other actors in the scenar
io as though they were people with integrity and real motivations.
For example, if an actor playing another clinician is angry and yellin
g, the debriefer would help participants think about exploring the c
haracter’s legitimate concern.
Positive, effective behaviors include:
• Stating that the physical properties of the simulator are
different from the real world.
10
• Stating that the participants likely act differently in the
simulated environment.
• Stating or demonstrating that despite simulation limitations, th
ere are still useful things to discuss (e.g., by comparing events an
d situations to real life).
• Acknowledging a participant’s issues or complaints by paraphrasi
ng and accepting them. (E.g., “You were frustrated with the breath
sounds. Yes, they are not great.”)
• After validating participants’ experience, directing discussion to fr
uitful areas. (E.g., “True, breath sounds can be difficult to discern o
n the mannequin so I sympathize with that. I wonder has anyone e
ver had trouble diagnosing breath sounds in real life and what that’
s like for you?”)
• Acknowledging that there’s a certain awkwardness in
“pretending.”
Negative, ineffective behaviors include:
•Arguing with trainee about realism.
• Denying the legitimacy of participants’ complaints.
• Showing resentment that participants did not fully engage in the
simulation.
• Laughing at or belittling the technology or actors in the
simulation.
• Making students feel foolish for entering fully into the fiction of
the case, procedure, or event.
Conveys respect for learners and concern for their
psychological safety.
Conveying respect and positive regard for participants helps creat
e a context conducive to the conversational probing needed for a
good debriefing. Instructors generate a favorable
environment when they treat participants as respected team mem
bers within a given profession. Communicating such respect can go
hand in hand with pointed critiques of a participant’s performance.
It does not require hiding one’s judgments. It does require asking q
uestions and respectfully listening to the answers.
Positive, effective behaviors include:
• Showing genuine curiosity by eliciting participants’ thoughts and
listening to the answers.
• Asking open-‐ended questions that encourage participants to
explore their thinking.
• Taking notes or remembering what participants say and making
connections during the debriefing about insights or explanations.
• Critiquing the behavior, not the person. (E.g., saying, “Shouting a
t the nurse didn’t get the response you needed,” instead of, “You’
re hot-‐headed, aren’t you?”)
Negative, ineffective behaviors include:
• Asking and answering one’s own questions.
• Talking over participants’ remarks.
• Demeaning participants or ridiculing their answers.
• Pursuing a line of questioning intended to make a participant
look or feel stupid.
• Using mean-‐spirited sarcasm or irony in discussing participants’
actions or thinking.
11
ELEMENT 3 Structures the debriefing in an organized way
While different schools of debriefing have different numbers of
phases, almost all approaches advocate a logical flow that allows
participants to 1) “blow off steam”—vent their immediate
reactions; 2) analyze what happened and generalize or apply to
their current or future practice; and 3) summarize their learning.
Therefore, an effective debriefing should begin by eliciting
reactions early on from participants; the middle should comprise
an understanding phase that includes both analysis and
generalizing to other settings; the debriefing should conclude with
a summary phase.
Element 3 Dimensions
Encourages trainees to express their reactions and, if
needed, orients them to what happened in the simulation,
near the beginning.
This phase early in the debriefing allows for participants to express
their initial emotional reactions to the simulation and, if needed, fo
r the instructor to provide information or facilitate a conversation t
hat clarifies the facts underlying the simulation. The instructor’s qu
estions and responses to what a participant says
can help or hinder the development of psychological safety. Invitin
g reactions and listening with interest helps create a sense of safet
y. The debriefer ascertains the participants’ understanding of the ca
se, procedure, or event and may clarify salient facts. The participan
ts’ comments provide hints about their concerns, and
informs the debriefer about what objectives are important to
address.
Positive, effective behaviors include:
• Asking questions that invite participants to express their initial re
actions to the case, procedure, or event, and allowing emotions to
be expressed.
• Accepting and encouraging reactions by nodding or
paraphrasing etc.
• Discussing facts and reviewing events when necessary to elimi
nate confusion expressed by the participant. This can also take
place in analysis phase.
Negative, ineffective behaviors include:
• Ridiculing participants’ responses to the experience.
• Becoming defensive about negative reactions participants may
express about the case, procedure, or event.
• Skipping over the reactions phase, which may leave the
participant feeling unsettled or unsafe.
Guides an analysis of trainees’ performance during the
middle of the session.
The purpose of the analysis phase is to allow participants to make
sense of simulation events, address their concerns, and help partici
pants move toward accomplishing course objectives. Instructors sh
ould attempt to elicit the thought processes and feelings that drov
e a participant's actions and then work with trainees to rethink or a
ugment these underlying drivers of performance to ensure better p
atient care in similar situations in the future. In this phase the debr
iefer also helps the participant see how lessons can be generalized
to other situations.
12
Positive, effective behaviors include:
• Asking questions that prompt the participant to discuss and refle
ct on what they thought, felt and what happened during the simul
ation.
• Listening to issues raised by the participant so that the inst
ructor can tailor short micro lectures to address learner con
cerns.
• Spending appropriate time on topics of high interest for trainees
while being sure to cover topics that are part of the course require
ments.
• Asking questions and provoking discussion that lead participants t
o speculate on how lessons learned in simulation might apply in th
eir healthcare setting—
in other words, helping participants generalize from the simulation
experience.
Negative, ineffective behaviors include:
• Immediately telling the participant what to DO differently next t
ime (e.g., “Call for help immediately; don’t wait!”) without elicitin
g their thinking about what and why they did.
• Running through a list of teaching points per the curriculum with
out regard to specific incidents—positive or negative—
during the case, procedure, or event from the trainees’ perspective
.
• Cutting off participants’ explanations of why they pursued a
certain course of action.
Collaborates with participants to summarize learning
from the session, near the end.
The purpose of the summary phase is to signal the end of the
debriefing, to review salient points, and to translate lessons le
arned from the debriefing into memorable principles that train
ees can take with them to improve their practice.
13
Positive, effective behaviors include:
• Stating that it is time for the discussion to close, and making
time for participants to summarize what they have learned.
• Asking participants a question or set of questions to help them su
mmarize what they have learned. (E.g., “What went well? Given si
milar circumstances, what might you do differently next time? Wha
t lessons will you use in your practice?”)
• Summarizing important points if the participants did not cover
them.
• Recommending reading or activities that participants can pursue
to improve.
Negative, ineffective behaviors include:
• Ending the debriefing abruptly with no summary of main learn
ing points (E.g., “Oh, look at the time! That’s all we’ve got time
for. Have a nice day.)”
• Monopolizing the summary discussion without giving trainees a
chance to contribute concluding remarks.
• Treating the debriefing as an isolated experience without
stressing its connection to past and future practice.
14
ELEMENT 4 Provokes engaging discussions
How well does the debriefer engage the participants in interesting
discussions, listen to what they say, and help them to be reflective
practitioners? The purpose of debriefing is to get participants to
focus on important topics and generate in-‐depth
discussion. Debriefings should not focus on simple elicitation of
knowledge and facts alone. Rather, good debriefings require
participants to apply, analyze, synthesize, and evaluate
information. The ultimate
goal of debriefing is to encourage participants to personally reflect
on their approach to clinical practice or healthcare management
and to inspire improvement.
Element 4 Dimensions
Uses concrete examples and outcomes as the basis for inquiry and discussion.
Examining the visible clinical, social, or teamwork actions taken by
the participant and the outcomes of the case, procedure, or event a
llows the debriefer and participant to work with verifiable, public da
ta as the starting point for discussion. This could include, for exampl
e, whether the patient coded or important information
was lost. From that starting point, debriefings explore the
participant’s personal “frames” that drove his or her actions.
Positive, effective behaviors include:
• Asking questions based on observed actions and results. “So it l
ooked to me like it took three minutes to defibrillate the patient
…”
• Using observations of behavior and interactions to launch discus
sion of thought processes or ways to improve action in the future.
“So it looked to me like it took three minutes to defibrillate the p
atient. In the last debriefing, we discussed the importance of shoc
king the patient quickly. This makes me think there is something
hard about getting this task accomplished. How do you all see it?”
Negative, ineffective behaviors include:
• Stating inferences or beliefs about trainee performance as fact (
e.g., “You didn’t know what the rhythm was,”) without acknowled
ging them as inferences and thereby subjecting them to correctio
n by participants.
• Harping on the outcome of the case, procedure, or event, parti
cularly if poor, without breaking down the action sequence for a
nalysis. Reveals own reasoning and judgments.
There is a long tradition of Socratic dialogue and questioning in he
althcare education. Too often this process is implemented in a wa
y that violates both the spirit and letter of Socratic dialogue, a proc
ess in which both the teacher and student’s thought processes are
meant to be open to question. Instead, healthcare instructors ofte
n ask a string of questions without revealing their own thinking. T
he result of hiding one’s own thinking is that trainees are often con
fused about why a question is being asked; worse, they may feel m
anipulated or unfairly trapped. Debriefers can avoid these problem
s by revealing their own reasoning or rationale for pursuing a line
of questioning, and doing so in a way that is curious and respectful
of the learner. Ideally, assumptions or conclusions a debriefer has
made about trainee performance are communicated so that they a
re open for adjustment by trainees. In other words, instructors sho
uld view their own
15
conclusions with healthy skepticism and assume the trainee is
well-‐intentioned and intelligent.
Positive, effective behaviors include:
• Describing one’s own reasoning to make clear why the topic is im
portant for the debriefing. E.g. “Even though chest compressions se
em like a relatively simple thing to do, people often have trouble wi
th it in the midst of a code. So, I’d like to discuss the challenges and
tricks for maintaining the right rate and getting good chest recoil.”
• Respectfully stating personal judgments or concerns about partici
pant performance (good, bad, unusual, interesting, alarming, etc.) s
o that the participant does not have to guess why the question is b
eing asked or feel confused about the debriefer’s point of view. E.g.
“I didn’t see you double-
‐check the dose of that medication. Without the double-
‐check I worry that the patient is at more risk of getting the wrong
dose. I’m wondering what was on your mind at the time?”
• Stating supporting information for the debriefer’s point of view,
such as personal experience, seeing others, reading, and so forth.
• Couching inferences or beliefs about trainee performance in ten
tative terms (e.g., instead of saying “You had no leader,” saying “I
t looked to me like you had trouble assigning a leader in this case.
”), and not assuming to know what the participant was thinking at
the time.
Negative, ineffective behaviors include:
• Keeping one’s own concerns or reasoning hidden.
• Asking leading questions that conceal a judgment or solution in t
hem. (E.g., “Wouldn’t it have been better to call for help much ear
lier?”)
• Framing questions so that they trap participants into displaying l
ack of knowledge or admitting mistakes without revealing the pur
pose of the question. (E.g. “When is a beta blocker like this indicat
ed?”)
• Stating one’s own conclusions as truth. E.g. “You were fixated. Y
ou didn’t consider anaphylaxis as a possible diagnosis.” Facilitates discussion through verbal and non-verbal
techniques.
Debriefers use verbal and non-
‐verbal techniques both consciously and unconsciously during debri
efing. This element relates to using these techniques to elicit and lis
ten to trainees’ input as well as to control discussion.
Positive, effective behaviors include:
• Involving everyone; not allowing one or two people to dominate
the discussion.
• Eliciting and utilizing differing viewpoints to enrich understanding
of a topic. “So, Frank it sounds like you think whether or not the pati
ent’s problem was iatrogenic shouldn’t influence adherence to adva
nce directives. But Elena, you think it should be taken into account.
Help me understand the thinking behind your positions?”
• Drawing in people who are quiet by asking them substantive qu
estions about the scenario. E.g. “Ravi, I saw you reviewing the pati
ent’s record, what kinds of things caught your attention?”
• Listening carefully to participant remarks without interrupting.
• Using body language such as head nods, eye contact, posture, pro
ximity and distance, standing or sitting, and facial expression to pos
e a challenge, or help show interest, kindness, or power, but all in th
e service of a productive discussion.
• Allowing silence to give the participant time to think about
questions.
• Paraphrasing, or verbally mirroring what trainees say.
16
Negative, ineffective behaviors include:
• Grimacing, rolling the eyes, tapping fingers, looking bored.
• Leaving no silent time so participants can speak up.
• Muttering under one’s breath.
• Interrupting or cutting trainees off repeatedly.
• Allowing some participants to dominate discussion or “playing
favorites” by showing more interest in them.
• Consistently allowing some trainees to be shut out of the
conversation.
Uses video, replay, and review devices (if available).
Video or other replay and review devices are an effective educat
ional technique and can be used to help participants see their ac
tions as they relate to key objectives of the debriefing.
Positive, effective behaviors include:
• Showing one or a few well-‐chosen short segments to illustrate
and introduce topics.
• Using replay as a springboard for discussion.
• Operating replay equipment efficiently (e.g., finding desired
segments with little delay).
• Linking playback to key objectives; using replay and review
devices to help make interesting points or to deepen a discussion.
• Pausing the replay if substantial discussion evolves.
Negative, ineffective behaviors include:
• Playing long pieces of video with no discussion or framing of the
purpose.
• Not referring to video at all during when its use might highlight
important points or resolve a conflict.
• Making fun of people’s appearance or sound on camera.
Recognizes and manages the upset participant.
Skip rating this dimension if no participants appear upset.
At its best, simulation is emotionally engaging. That being the case, t
here are naturally times when a participant may become upset. The
skilled debriefer will help the participant clear the air and help the gr
oup get back to an emotionally stable state. Even a moderately upse
t participant will allow for consideration of this dimension.
Positive, effective behaviors include:
• Noticing and recognizing when someone becomes upset; stating
this as an observation and respectfully checking with the participan
t whether their observation is accurate.
• Inviting or allowing the participant to describe feelings—if the
participant wishes.
• Trying a variety of techniques to re-
‐establish equilibrium, from normalizing the behavior (e.g., if a parti
cipant is upset about her performance, putting it in the context of p
erformance of others in similar simulations (such as saying, “We’ve
done this scenario 40 times and almost everyone handles it the sam
e way you did”), allowing other participants to defend or bolster a f
ellow participant, etc.
• Dealing openly with the root of the emotional upset.
• Guiding the discussion and timing to help the participant confront
and resolve the upset, possibly by changing the pace of the discussio
n; distributing the conversation to others to explore their
17
perspective or their contribution to the troublesome issue instead
of focusing exclusively on the upset participant; retreating to less e
motional material but coming back to the difficult issue either withi
n the debriefing or afterwards privately with the trainee.
Negative, ineffective behaviors include:
• Ignoring, hounding, or ostracizing an upset participant.
• Laughing at or belittling the reason for a person’s being upset.
• Creating rivalries among participants over issues of
disagreement.
18
ELEMENT 5 Identifies and explores performance gaps
In rating this Element, consider how well the debriefer describes the
performance gap─that is, the difference between the actual
performance and the optimal performance. Debriefings should
provide participants concrete feedback about performance. Beyond
identifying it, the debriefer should explore the causes of the gap.
When performance is sub-‐optimal, the debriefer works with the
participant to analyze how the trainee’s “frames” (including
knowledge and attitudes) and skills led to the performance gap. If a
student does well, it doesn’t mean there’s nothing to talk about: In
the event that performance was good or excellent, the instructor
assists the participant in identifying the knowledge, skills, and
attitudes that contributed to that good performance.
Element 5 Dimensions
Provides feedback on performance.
When learning a complex skill, learners often lack a clear sense of
how they are progressing—
just like a person first learning to drive may have trouble determini
ng where he is in the lane. Learners need feedback, and that usuall
y means highlighting the gap between of where they want to be an
d where they are. Clear feedback—critiques—
about how participant performance either falls short of, meets, or
exceeds the desired performance for the simulation course is crucia
l for learning.
Positive, effective behaviors in this dimension include:
• Stating judgments or critiques of participants’ performance in a
specific—and actionable—
form . (E.g., Instead of saying, “Communication was poor,” saying,
“I didn’t see you meet the patient’s eye or paraphrase what she sai
d.”)
• Commenting on positive as well as negative performance.
• Being direct yet respectful. (E.g. “I didn’t hear you apologize to th
e patient for the mistake. I think this contributed to his getting ang
ry”; e.g. “I didn’t see you double check the dose. I’m worried that
increases the risk of a medication error.”)
Negative, ineffective behaviors include:
•Failing to reveal opinion or judgment when describing
observations about the trainee’s performance.
• Disguising criticism by sugar-
‐coating, camouflaging, or using the “sandwich technique” (positive
statement followed by criticism followed by a positive statement).
• Asking leading questions in hopes the participant will state the p
erformance deficit the instructor observed but does not want to sa
y.
• Expressing judgments in a sarcastic, hurtful, insulting, or
disrespectful way (e.g. “It’s no wonder the patient started s
creaming! What part of ‘I’m sorry’ is so hard to say?!”)
Explores the source of the performance gap.
Exploring the reason or driving force for trainees being where
they are instead of where they may have been expected to be is a
distinctive feature of a good debriefing. It is useful to help particip
ants understand how their frames (such as knowledge, assumption
s, beliefs, or feelings) contributed to their performance. Debriefers
should help participants explore these foundations of their actions.
19
Occasionally, it is necessary because
20
of time constraints or sufficient because of course objectives to f
ocus only on correcting actions rather than helping participants e
xplore and rethink their frames. A debriefing that focuses exclusiv
ely on actions will usually not be rated above a 4 and rarely recei
ve a 5 on this Element.
Positive, effective behaviors include:
• Going beyond the “what” to the “why;” that is, exploring why par
ticipants took the actions they did without focusing exclusively on t
he do’s and don’ts of participant actions.
• Encouraging participants to reflect on what they were thinking
at the time.
• Providing specific, actionable critiques. E.g. “I didn’t hear you su
mmarize the mother’s concerns.” Or “You had your shoulders hunc
hed near your ears—see here on the video?—
I think this might have contributed to the awkwardness and fatigue
you felt during that laparoscopy. Try dropping them down into the s
ockets next time.”
• When appropriate to the goals of the course, helping the partici
pant explore and understand behavioral issues
that contributed to a performance gap, such as communication st
yle, planning, workload management etc.
• When appropriate to the goals of the course, helping the particip
ant understand the kinesthetic or psychomotor issues that contribu
ted to the performance gap, such as inadequate depth of chest co
mpressions, or poor suturing technique during an intestinal anasto
mosis.
• When appropriate to the goals of the course, helping the partici
pant understand how their clinical knowledge and assumptions or
their application contributed to the performance gap, such as wha
t were the causes of the delayed identification and treatment of a
n air embolism.
Negative, ineffective behaviors include:
• Focusing only on correcting actions.
• Shutting down discussion of participants’ thought processes.
• Utilizing shallow or too-
‐abstract characterizations of behavior. (e.g. “You didn’t listen to t
he mom.” E.g. “You need to relax your shoulders more so you don
’t get tired during a long procedure.”)
• Sneering at a trainee’s assumptions, faulty knowledge, or
physical coordination.
21
ELEMENT 6 Helps trainees achieve or sustain good future performance
Rating this Element requires gauging how effectively the debriefer
helps learners align hypothetical future performance in line with
expectations or to repeat good or excellent performance.
Debriefings should assist participants in developing the knowledge,
skills, and attitudes to close any gaps between the level of
performance the instructor desires and what was observed. In the
event that the performance was good or excellent, the instructor
should identify the good behaviors or responses that made it
especially successful and elicit the underlying thought processes
that enabled it. Helping students become aware of what actions or
frames work well helps sustain good performance. The skilled
debriefer is knowledgeable in the subject area and is able to work
with performance gaps revealed in the simulation to generate
discussions about how to improve or maintain clinical or teamwork
performance in other settings or in the future.
Element 6 Dimensions
Helps close the performance gap through discussion and
teaching.
Once the basis of a performance gap is understood, it is time to he
lp participants understand how to perform more effectively next ti
me. The approach to closing the performance gap can be done by
discussing changes in frames and actions (usually resulting in highe
r DASH scores for the debriefer), or just changes in actions.
Positive, effective behaviors include:
• Discussing the current case, procedure, or event to elicit—from
trainees—ways to improve future performance.
• If student do not generate adequate new practices through disc
ussion, then using a very short lecture to share knowledge, experi
ence, or research findings that inform participants how to improv
e performance.
• Providing specific information or knowledge whose absence
contributed to the performance gap.
• Using information a participant has shared to help arrive at new
ways to think about and solve clinical or teamwork problems.
• Stressing that the main concern is patient well-‐being.
Negative, ineffective behaviors include:
• Not suggesting or eliciting ways to improve performance in
future practice.
• Not backing up instruction with explanatory knowledge.
• Sarcastically telling trainees they should consider another
career.
• Ignoring good or outstanding performance.
Demonstrates firm grasp of the subject.
Effective debriefers have expertise in the subject area. Topics in de
briefings may involve issues that have to do with the clinical situati
on, behavior, teamwork, ethics, etc. The debriefer discusses topics
knowledgeably without overstepping the boundaries of expertise.
19
Positive, effective behaviors include:
• Imparting knowledge that is reasoned and current.
• Encouraging correct practices and knowledgeably elaborating
on underlying principles.
• Being obviously comfortable with the material being discussed
and checking to make sure the participant understands.
• Informing participants when the limits of the debriefer’s
knowledge are reached.
• Being interested in eliciting the best knowledge rather than
maintaining an appearance of knowing everything.
Negative, ineffective behaviors include:
•Providing outdated or incorrect information.
• Allowing unacceptable practices to go unaddressed.
• Not being open to learning from others, including students.
[Note: Serious oversights or egregiously wrong information
imparted by the instructor is cause to rate this entire Element as a
1 even if other Dimensions within Element 6 are done well.]
Meets the important objectives of the session.
The rater must know the objectives to use this dimension.
Simulated events, procedures or other activities take place within a
larger curriculum and have specific educational objectives. The de
briefer meets the goals of the curriculum. If objectives are discusse
d other than the intended ones, the debriefer appears to make rati
onal choices.
Positive, effective behaviors include:
• Discussing all the important educational objectives of the
scenario.
• Deviating from the curriculum when there is high educational
benefit or backtracking to more fundamental building blocks to
meet higher level or critically important objectives.
• Discussing objectives in such a way as to demonstrate comfort
and expertise with relation to the objectives.
• Checking to ensure that participants understand the principle,
technique, or approach under discussion.
• If objectives are not met, talking about how or why that
happened.
Negative, ineffective behaviors include:
•Allowing the conversation to meander onto topics not crucial to
curriculum.
• Allowing key learning points to be missed.