enhancing veterinary students’ clinical decision …
Post on 20-Apr-2022
4 Views
Preview:
TRANSCRIPT
ENHANCING VETERINARY STUDENTS’ CLINICAL DECISION-MAKING SKILLS BY
PROMOTING REVISION OF THEIR DECISION-MAKING PROCESS IN CASE-BASED
LEARNING
by
HYOJIN PARK
(Under the Direction of Ikseon Choi)
ABSTRACT
Despite the importance of clinical decision-making, recent veterinary graduates felt that
their clinical decision-making skills were so minimal that they were unable to complete their
work independently. In order to enhance veterinary students’ clinical decision-making skills,
two instructional supports—a case-based online learning module and scaffolded revision
activities—were implemented based on a case-based learning model proposed by Choi and his
colleagues (2013) and findings from other research. To elaborate, the case-based online learning
module was utilized to enhance veterinary students’ knowledge application by providing realistic
context, and scaffolded revision activities were utilized to promote reflective thinking by
providing students with an opportunity to compare their opinions to those of experts and/or
peers.
Forty-seven out of one-hundred-two veterinary junior students who enrolled in a small
animal digestive disease course participated in this study. The participants were allowed to self-
select between three scaffolded revision activity groups: expert commentary only, expert
commentary as well as early peer feedback, and expert commentary as well as later peer
feedback. Quantitative data and qualitative data were collected from students’ initial and revised
clinical decisions, a transfer decision-making test, an online survey, and face-to-face interviews.
The quantitative results indicated that the scaffolded activity with expert commentary
was helpful in enhancing the quality of the participants’ revised clinical decisions. However, the
peer feedback and its timing did not influence the quality of the revised clinical decisions.
Furthermore, the results of the transfer test showed that there was no statistically significant
difference among the three groups.
The qualitative results based on the online survey and face-to-face interviews provided
further insights that students could potentially benefit from the expert commentary in solidifying
their clinical knowledge and facilitating reflection upon their decision-making process.
Furthermore, the participants who received peer feedback felt that it helped them retain
knowledge better by allowing them to communicate their thoughts with peers.
INDEX WORDS: Clinical decision-making, Case-based learning, Knowledge application,
Reflection, Experts, Peer feedback
ENHANCING VETERINARY STUDENTS’ CLINICAL DECISION-MAKING SKILLS BY
PROMOTING REVISION OF THEIR DECISION-MAKING PROCESS IN CASE-BASED
LEARNING
by
HYOJIN PARK
B.A., Ewha Womans University, Republic of Korea, 2008
M.A., Ewha Womans University, Republic of Korea, 2010
A Dissertation Submitted to the Graduate Faculty of The University of Georgia in Partial
Fulfillment of the Requirements for the Degree
DOCTOR OF PHILOSOPHY
ATHENS, GEORGIA
2016
© 2016
HYOJIN PARK
All Rights Reserved
ENHANCING VETERINARY STUDENTS’ CLINICAL DECISION-MAKING SKILLS BY
PROMOTING REVISION OF THEIR DECISION-MAKING PROCESS IN CASE-BASED
LEARNING
by
HYOJIN PARK
Major Professor: Ikseon Choi
Committee: Michael Orey
Chad Schmiedt
Electronic Version Approved:
Suzanne Barbour
Dean of the Graduate School
The University of Georgia
May 2016
iv
DEDICATION
I dedicate this dissertation to my beloved family for supporting me with affections and
love.
v
TABLE OF CONTENTS
Page
LIST OF TABLES ........................................................................................................................ vii
LIST OF FIGURES ..................................................................................................................... xiii
CHAPTER
1 INTRODUCTION .........................................................................................................1
Problem Statement ...................................................................................................2
Research Focus ........................................................................................................3
Research Questions ..................................................................................................5
2 LITERATURE REVIEW ..............................................................................................7
Clinical Decision-Making ........................................................................................7
Knowledge Application .........................................................................................16
Reflection ...............................................................................................................22
Conceptual Framework ..........................................................................................31
3 METHODS ..................................................................................................................35
Participants .............................................................................................................35
The Small Animal Digestive Diseases Course ......................................................37
Case-Based Online Learning Module ....................................................................37
Scaffolded Revision of the Initial Clinical Decision .............................................45
Procedures ..............................................................................................................47
Research Design.....................................................................................................49
vi
Data Collection and Analysis.................................................................................52
4 RESULTS ....................................................................................................................63
RQ1. Revision effects of the scaffolded revision activities ...................................63
RQ2. Revision effects by groups ...........................................................................73
RQ3. Revision effects by groups across sessions ..................................................91
RQ4. Transfer effects ...........................................................................................109
RQ5. Student perception on the revision experiences .........................................115
5 CONCLUSION ..........................................................................................................129
Summary of the Findings .....................................................................................130
Effects of the case-based online learning module and scaffolded revision .........136
Implications of the Study .....................................................................................140
Suggestions for future research ............................................................................142
REFERENCES ............................................................................................................................144
APPENDICES
A PEER FEEDBACK GUIDELINES WITH REFLECTIVE PROMPTS ...................159
B ONLINE SURVEY ....................................................................................................163
vii
LIST OF TABLES
Page
Table 2-1: Characteristics of Type 1 and Type 2 decision-making approaches ........................... 10
Table 3-1: Participants’ gender, averaged GPA, and the result of Levene’s Test ........................ 36
Table 3-2: The question prompts embedded at Decision Point 1 in the learning module ............ 43
Table 3-3: A time series design for this research .......................................................................... 52
Table 3-4: The research questions, data collection, data source, and data analysis techniques ... 53
Table 3-5: Three dimensions of the clinical decision-making skills and their rubric ................... 56
Table 3-6: Sample ideal script at Decision Point 2 “Taking Action for Doug” ........................... 58
Table 3-7: Inter-rater reliabilities over the Decision Points .......................................................... 59
Table 4-1: Data collection for the quality of student decision-making across groups in three
sessions ............................................................................................................................. 63
Table 4-2: Data used to test the revision effect (Research Question 1) ........................................ 65
Table 4-3: Descriptive statistics on the quality of the initial and revised case assessment .......... 66
Table 4-4: Summary of repeated-measures ANOVA for the overall quality of the initial and
revised clinical decisions .................................................................................................. 67
Table 4-5: Descriptive statistics on the quality of the initial and revised case assessment .......... 68
Table 4-6: Summary of repeated-measures ANOVA for the qualities of the initial and revised
case assessment ................................................................................................................. 69
Table 4-7: Descriptive statistics on the quality of the initial and revised prioritization of issues
and objectives.................................................................................................................... 70
viii
Table 4-8: Summary of repeated-measures ANOVA for the qualities of the initial and revised
prioritization of issues and objectives ............................................................................... 71
Table 4-9: Descriptive statistics on the quality of the initial and revised plan of an immediate
action ................................................................................................................................. 72
Table 4-10: Summary of repeated-measures ANOVA for the quality of the initial and revised
plan of an immediate action .............................................................................................. 72
Table 4-11: Data used to test the two-way interaction effect of revision and group (Research
Question 2) ........................................................................................................................ 75
Table 4-12: Descriptive statistics on the overall quality of the initial and revised clinical
decisions among EC/NP, EC/EP, and EC/LP ................................................................... 76
Table 4-13: Summary of repeated-measures ANOVA for the overall quality of the initial and
revised clinical decisions in EC/NP, EC/EP, and EC/LP ................................................. 77
Table 4-14: Descriptive statistics on the quality of the initial and revised case assessment among
EC/NP, EC/EP, and EC/LP ............................................................................................... 78
Table 4-15: Descriptive statistics on the quality of the initial and revised prioritization among the
EC/NP, EC/EP, and EC/LP ............................................................................................... 79
Table 4-16: Descriptive statistics on the quality of the initial and revised plan among EC/NP,
EC/EP, and EC/LP ............................................................................................................ 80
Table 4-17: Descriptive statistics of the overall quality of the initial and revised clinical decisions
between peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP) ....... 82
Table 4-18: Summary of repeated-measures ANOVA on the overall quality of the initial and
revised clinical decisions in the peer feedback (EC/EP and EC/LP) and no peer feedback
groups (EC/NP) ................................................................................................................. 83
ix
Table 4-19: Descriptive statistics on the quality of the initial and revised case assessment
between peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP) ....... 84
Table 4-20: Descriptive statistics on the quality of the initial and revised prioritization between
peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP) ..................... 85
Table 4-21: Descriptive statistics of the quality of the initial and revised plan between peer
feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP) ............................. 86
Table 4-22: Descriptive statistics on the overall quality of the initial and revised clinical
decisions between EC/EP and EC/LP ............................................................................... 87
Table 4-23: Summary of a repeated-measures ANOVA for the overall quality of the initial and
revised clinical decisions in the EC/EP and EC/LP groups .............................................. 88
Table 4-24: Descriptive statistics of the qualities of the initial and revised case assessment
between EC/EP and EC/LP ............................................................................................... 89
Table 4-25: Descriptive statistics on the quality of the initial and revised prioritization between
EC/EP and EC/LP ............................................................................................................. 90
Table 4-26: Descriptive statistics of the qualities of the initial and revised plan between expert
commentary with early peer feedback group (EC/EP) and expert commentary with later
peer feedback group (EC/LP) ........................................................................................... 91
Table 4-27: Data used to test the three-way interaction effect (Revision x group x session)
(Research Question 3) ....................................................................................................... 92
Table 4-28: Descriptive statistics of the overall quality of the initial and revised clinical decisions
among the EC/NP, EC/EP, and EC/LP across two sessions ............................................. 93
Table 4-29: Summary of a repeated-measures ANOVA for the overall quality of the initial and
revised clinical decisions among the EC/NP, EC/EP, and EC/LP .................................... 95
x
Table 4-30: Descriptive statistics of the overall quality of the initial and revised case assessment
among EC/NP, EC/EP, and EC/LP across two sessions ................................................... 96
Table 4-31: Descriptive statistics of the overall quality of the initial and revised prioritization of
issues and objectives among the EC/NP, EC/EP, and EC/LP across two sessions .......... 97
Table 4-32: Descriptive statistics of the overall quality of the initial and revised plan of an
immediate action among EC/NP, EC/EP, and EC/LP across two sessions ...................... 98
Table 4-33: Descriptive statistics of the overall qualities of the initial and revised clinical
decisions between peer feedback (EC/EP and EC/LP) and no peer feedback groups
(EC/NP) across two sessions ............................................................................................ 99
Table 4-34: Summary of repeated-measures ANOVA for the quality of the initial and revised
case assessment in the peer feedback and no peer feedback groups ............................... 100
Table 4-35: Descriptive statistics of the quality of the initial and revised case assessment between
peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP) across two
sessions ........................................................................................................................... 101
Table 4-36: Descriptive statistics on the quality of the initial and revised prioritization of issues
and objectives between peer feedback (EC/EP and EC/LP) and no peer feedback groups
(EC/NP) across two sessions .......................................................................................... 102
Table 4-37: Descriptive statistics on the quality of the initial and revised plan of an immediate
action between peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC)
across two sessions ......................................................................................................... 103
Table 4-38: Descriptive statistics on the overall quality of the initial and revised clinical
decisions between EC/EP and EC/LP across two sessions ............................................. 104
xi
Table 4-39: Summary of repeated-measures ANOVA for the quality of the initial and revised
case assessment in EC/EP and EC/LP ............................................................................ 106
Table 4-40: Descriptive statistics on the quality of the initial and revised case assessment
between early peer feedback group and later peer feedback group across two sessions 107
Table 4-41: Descriptive statistics of the quality of the initial and revised prioritization of issues
and objectives between early peer feedback group and later peer feedback group across
two sessions .................................................................................................................... 108
Table 4-42: Descriptive statistics on the quality of the initial and revised plan of an immediate
action between early peer feedback group and later peer feedback group across two
sessions ........................................................................................................................... 109
Table 4-43: Data used to test the transferred effects of the scaffolded revision activities
(Research Question 4) ..................................................................................................... 110
Table 4-44: Descriptive statistics on the scores on the transfer test among EC/NP, EC/EP, and
EC/LP .............................................................................................................................. 111
Table 4-45: Summary of One-way ANOVA for the scores on the transfer test EC/NP, EC/EP,
and EC/LP ....................................................................................................................... 112
Table 4-46: Descriptive statistics of the scores on the transfer test between the peer feedback
groups (EC/EP and EC/LP) and no peer feedback group (EC/NP) ................................ 113
Table 4-47: Summary of One-way ANOVA for the scores on the transfer test between groups
with peer feedback (EC/EP and EC/LP) and without peer feedback (EC/NP)............... 113
Table 4-48: Descriptive statistics of the scores on the transfer test between EC/EP and EC/LP 114
Table 4-49: Summary of One-way ANOVA for the scores on the transfer test between groups
with early peer feedback (EC/EP) and with later peer feedback (EC/LP) ...................... 115
xii
Table 4-50: Means and standard deviations of the items about student perceptions on the
scaffolded-revision activity with expert commentary ..................................................... 117
Table 4-51: Means and standard deviations of the items about student perceptions on the
scaffolded revision with peer feedback ........................................................................... 119
xiii
LIST OF FIGURES
Page
Figure 2-1: Conceptual framework for this study to enhance veterinary students’ clinical
decision-making skills ..................................................................................................... 32
Figure 3-1: A sample screen of an initial decision-making activity in the case-based learning
environment ..................................................................................................................... 38
Figure 3-2: A sample screen of a revision activity in the case-based learning environment ........ 39
Figure 3-3: Five Decision Points in the module ........................................................................... 41
Figure 3-4: Scoring system for student decision-making responses ............................................. 56
Figure 3-5: A sample question from the final test ........................................................................ 60
Figure 4-1: The overall quality of initial and revised clinical decisions ....................................... 66
Figure 4-2: The quality of the three sub-dimensions of the initial and revised clinical decisions 73
1
CHAPTER 1
INTRODUCTION
Decision-making refers to the process of making a choice between a number of options,
which requires the decision maker to handle data and algorithms to decide a best choice
(Jonassen, 2010; Patton, 1978; Smith, Higgs, & Elizabeth Ellis, 2008; Thomas, Wearing, &
Bennett, 1991). The level of certainty surrounding a decision can vary, but most real world
decisions are made under uncertainty (LeBoeuf & Shafir, 2005). Such uncertain decisions are
ambiguous in that the decision maker must estimate the likelihoods of possible outcomes, which
are not known (LeBoeuf & Shafir, 2005).
Decision-making plays a central role in everyday life as well as in many academic
disciplines including medical fields (LeBoeuf & Shafir, 2005). On top of that, veterinary
practice often involves making decisions under uncertain circumstances. To elaborate, clinical
decisions have vaguely defined or unclear goals and unstated constraints (Jonassen, 1997;
Mamede & Schmidt, 2004; Maudsley & Strivens, 2000; Orasanu & Connolly, 1993; Patton,
1978; Terry & Higgs, 1993). In addition, the decisions are often made in a world of incomplete
and imperfect resources (Jonassen, 1997; Orasanu & Connolly, 1993) while multiple players
with different roles are involved in the act of decision making (Higgs & Jones, 2008; Orasanu &
Connolly, 1993). As veterinary practice has become more and more complex, the task of
decision-making has become more demanding for veterinarians.
2
Problem Statement
Despite the importance of decision-making, recent graduates felt that their clinical
decision-making skills were so minimal that as new qualified surgeons, they were unable to
complete their work independently (May, 2013; Vandeweerd et al., 2012b). Through literature
review, two major issues that veterinary students have experienced while developing decision-
making skills were identified: lack of knowledge application skills and reflective thinking skills.
In terms of the knowledge application issue, it seems that many veterinary students have
difficulties with applying their academic knowledge acquired from classroom settings to actual
real world problems (J. S. Brown, Collins, & Duguid, 1989; Gee, 1997). This issue seems to be
stemmed from the dichotomized curriculum and their goals (Maudsley & Strivens, 2000; Spiro,
Coulson, Feltovich, & Anderson, 1988). To elaborate, in introductory learning, students in
medical fields are exposed to a body of knowledge from various subject areas of biological
science and expected to establish their own knowledge structure (Maudsley & Strivens, 2000;
Spiro et al., 1988). This extensive amount of basic knowledge is often context-independent and
oversimplified due to the superficial similarities among related phenomena (J. S. Brown et al.,
1989; Spiro et al., 1988). At an advanced knowledge acquisition level, on the other hand, they
are expected to apply the basic knowledge from formal instruction to real settings (Maudsley &
Strivens, 2000; Spiro et al., 1988). Obstacles to application of the unsaturated knowledge to real
or realistic contexts would be obvious.
In addition to the knowledge application skills, reflective thinking skills are another
important variable in enhancing the quality of clinical decisions (Mamede & Schmidt, 2005;
Mamede, Schmidt, & Penaforte, 2008). Many studies have shown that experienced clinicians
put more time and effort on reflection than inexperienced clinicians (e.g., Chi, Glaser, & Rees,
3
1982; Patel & Groen, 1991). Reflection refers to deliberately thinking upon a past experience
with the intent of improving aspects of said experience (Schon, 1988). Reflection is believed to
facilitate deeper learning and lessen the gaps between theory and practice by allowing users to
engage in the critical thinking processes (Forneris 2004) and subjective or objective
interpretation of their experiences (Schon, 1988). In particular, reflection plays a key role in
leading students’ successful clinical decision-making by assisting students’ planning, monitoring,
and evaluating the course of actions (Bransford, Brown, & Cocking, 2000; Lin, Hmelo, & Kinzer,
1999; Shin et al., 2003) and protecting against errors (Higgs & Jones, 2008).
Research Focus
To enhance students’ clinical decision-making skills, proper instructional supports should
be provided (Collins, Brown, & Holum, 1991; Collins, Brown, & Newman, 1987). In this study,
students were asked to perform two learning activities: making a series of clinical decisions
through the exploration of a realistic case (which is referred as initial decision-making) and
revising the decisions (which is referred to revised decision-making). The initial decision-
making activity, in particular, was proposed to promote students’ knowledge application, and the
revision activity was identified to stimulate their reflective thinking. To better guide these
decision-making activities, two instructional supports—a case-based online learning module and
scaffolded revision activities—were proposed based on Choi’s case-based learning model (Choi,
Hong, Park, & Lee, 2013; Choi, Lee, & Kang, 2009; Choi & Lee, 2009; Choi, 2009) and
findings from other research on case-based learning, reflection, and decision-making.
A case-based online learning module was designed and developed to guide the initial
decision-making activity (Choi et al., 2013, 2009; Choi & Lee, 2009; Choi, 2009). Case-based
learning, which enables students to interpret, reflect on, and apply their direct or indirect
4
experiences to real or realistic contexts, is advocated as one of the promising teaching methods to
enhance students’ knowledge application skills as well as reflective thinking skills (Ertmer &
Russell, 1995). To elaborate, students are expected to lessen the gaps between theories and
practices by identifying real or realistic contexts, activating and elaborating on their context-
independent prior knowledge, and synthesizing as well as applying the knowledge to the real or
realistic contexts (Ertmer & Russell, 1995). During these activities in case-based learning,
students are also encouraged to reflect on their learning processes (Epstein, 1999; Mamede &
Schmidt, 2005).
The module employed for this study provided students with a series of realistic case
videos and critical thinking prompts (Choi et al., 2013, 2009; Choi & Lee, 2009; Choi, 2009).
Students were asked to watch the case videos, identify and analyze the problems, and make a
decision with the aid of critical thinking prompts. In order to create a realistic and educationally
valuable case, experienced clinical faculty developed a typical case of canine digestive disease
based on their past experiences. The case includes an entire cycle of clinical decision-making
activities, ranging from diagnosing a patient’s problem and announcing a treatment plan to
reacting to post treatment scenarios. The critical thinking prompts comprised of four questions
guiding each student’s decision-making process, and included the following four phases of the
decision-making process: identifying key information, assessing the case, prioritizing issues and
objectives, and making an immediate plan.
Then to support the revision of the initial decision, two scaffolding strategies for revision
were proposed and developed (J. S. Brown et al., 1989; Williams, 1992). It is believed that one
of the assuring methods of promoting individuals’ reflection is providing an opportunity for them
to compare their opinions to those of others (J. S. Brown et al., 1989; Williams, 1992). Based on
5
this finding, expert commentary videos and peer feedback were proposed to encourage students
to compare their clinical decisions to those of others. In the expert commentary videos, a team
of experienced veterinary educators shared their own approaches to the realistic videos. For peer
feedback, students had an opportunity to compare their initial decision with those of their peers
and provide feedback to each other.
These expert commentary videos and peer feedback were provided between the initial
decision and revising it. The expert commentary videos were embedded in the case-based
learning module and provided to all students, while peer feedback was provided in two optional,
separate sessions. Thus, all participants were allowed to participate in one of the three
scaffolded revision activities: expert commentary with no peer feedback (EC/NP), expert
commentary with early peer feedback (EC/EP), or expert commentary with later peer feedback
(EC/LP).
Research Questions
The purpose of this study was to examine the effects of the scaffolded revision activities
on the quality of the students’ revised clinical decisions. To elaborate, the gain effects and the
near-transferred effects of the scaffolded revision activities were investigated. Students’
perceptions on the scaffolded revision activities were also explored. The research questions that
addressed in the current study are as followed:
Research Question 1. Do the scaffolded revision activities enhance the quality of students’
revised clinical decisions in case-based learning?
Research Question 2. Do the scaffolded revision activities enhance the quality of the
different groups’ revised clinical decision in case-based learning?
6
Research Question 3. Are there significant differences in the quality of the initial and
revised clinical decision among groups across the two peer feedback sessions?
Research Question 4. Does the participation in a scaffolded revision activity affect
students’ transferred clinical decision-making skills?
Research Question 5. What are the students’ perceptions on the revision activities with
the case-based online learning module?
7
CHAPTER 2
LITERATURE REVIEW
This chapter provides a literature review of the relevant theoretical, conceptual, and
practical research used to inform this study. In particular, the first section provides an overview
of available decision-making theories. Next, the second and third sections discuss the two
typical difficulties students have in making a good clinical decision—knowledge application and
reflective thinking. Lastly, the fourth section presents a conceptual framework to enhance
veterinary students’ clinical decision-making skills through promotion of knowledge application
and reflection.
Clinical Decision-Making
Decision-making refers to a process of making a choice between a number of options,
which requires a decision maker to handle data and algorithms to decide a best choice (Jonassen,
2010; Patton, 1978; Smith et al., 2008; Thomas et al., 1991). Decision-makers should identify
the most viable option among many to the problem under the circumstances in which the
problem occurs (Jonassen & Hung, 2008). Clinical decision-making can also be defined as a
systematic approach to choose a best course of action between alternatives in clinical settings
(Banning, 2007; Patton, 1978; Thompson & Dowding, 2002).
Clinical practice comprises a series of decision-making performances (Cockcroft, 2007;
McKenzie, 2014; Patel, Arocha, & Zhang, 2005). Take an example of a doctor who needs to
diagnose a patient’s symptoms and plan the treatment. The doctor needs to decide which
questions to ask during the history taking and which systems to focus on during the physical
8
examination. Based on the data, the doctor determines which diagnostic tests to consider and
then which treatment plans to pursue. Lastly, the doctor is tasked with evaluating the outcome
and adjusting the diagnosis and/or treatment based on the patient's response to therapy or any
new information that has become available (Anene, 2013; Eddy, 1990).
Characteristics of Clinical Decision-Making
Clinical decision-making begins with a problem or a state of discrepancy which needs a
solution (Jenkins, 1985). The problem is often an uncertain, complex, difficult, and ill-defined
task with four characteristics as follows (Mamede & Schmidt, 2004; Maudsley & Strivens, 2000).
First, clinical decisions have vaguely defined or unclear goals and unstated constraints
(Jonassen, 1997; Orasanu & Connolly, 1993; Patton, 1978; Terry & Higgs, 1993). A clinician
may be driven by multiple purposes, and some of the purposes may be vague or conflict with
others (Orasanu & Connolly, 1993). For example, although a clinician would like to save a
patient using all possible treatment plans, the treatment plan may have risks for a particular
patient, or a choice of the treatment plans may be restricted due to the owners’ financial concerns.
Sometimes, organizational goals and norms may affect the decision (Higgs & Jones, 2008).
These conflicts are especially tricky in naturalistic decision-making settings, because the
dynamic situation may bring new values (Orasanu & Connolly, 1993). Since the situation may
continuously change, results with a decision can be ambiguous or sometimes risky (LeBoeuf &
Shafir, 2005).
Second, multiple players with different roles are involved in the act of decision-making
(Higgs & Jones, 2008; Orasanu & Connolly, 1993; Smith et al., 2008; Terry & Higgs, 1993).
Many of the clinical decisions involve parties, including a primary clinician, multiple specialists,
owners, agents of owners, and hospital administration (Orasanu & Connolly, 1993; Vandeweerd
9
et al., 2012a). The involvement of multiple players makes decision-making process more
complex, because it is hard to have all of the players share the same understanding of goals,
situational status, and decisions (Orasanu & Connolly, 1993).
Third, there is no absolute solution in ill-defined settings, and sometimes multiple or no
solutions exist (Jonassen, 1997; Kitchener, 1983). In clinical decision-making, no single
absolute treatment plan exists, and an appropriate treatment plan for a patient may not work for
another patient. In addition, although a clinician determines his/her solution, multiple players,
who described previously, may not agree with the solution, which can lead to no consensual
agreement.
Fourth, clinical decisions, in most cases, should be made in a world of incomplete and
imperfect resources (Jonassen, 1997; Orasanu & Connolly, 1993). Available resources may be
ambiguous, or sometimes its validity can be suspect (Orasanu & Connolly, 1993). When making
a clinical decision, a clinician often refers to colleagues as a primary resource consulted, because
doing so can serve as one of the quickest ways to obtain information necessary for his/her
decision-making (Vandeweerd et al., 2012b). However, colleagues sometimes may provide
conflicting resources. Also, patients or clients sometimes may incorrectly describe their
experiences, or diagnostic tests may leave open a range of possible diseases.
Decision-Making Theories
Dual process theory. The dual process theory, the most recent theory that explains the
process of decision-making, identifies that humans frequently use two systems to process
information and make a decision. Each system is called type 1 (system 1) and type 2 (system 2)
respectively (Chaiken & Trope, 1999; Croskerry, 2009; Evans, 2003; Evans & Over, 1996;
Hammond, 1996; Kahneman & Frederick, 2005). The two systems work in different
10
mechanisms: Type 1 users analyze information and make a decision on the basis of heuristics,
whereas type 2 users complete tasks using learned decision-making techniques (McKenzie,
2014). Table 2-1 lists the characteristics of type 1 and type 2 decision-making approaches.
Table 2-1
Characteristics of Type 1 and Type 2 decision-making approaches
Characteristics Type 1 (or System 1) Type 2 (or System 2)
In medical areas Pattern recognition Hypothetico-deductive reasoning
Type of reasoning Experiential-inductive Hypothetico-deductive
Cognitive style Heuristics Systematic
Cognitive awareness Unconscious Deliberate, purposeful
Conscious control Low High
Automaticity High Low
Rate Fast Slow
Errors Normative distribution Few but significant
Effort Low High
Emotional valence High Low
Influence of context High Low
Adapted from Croskerry, P., & Norman, G. (2008). Overconfidence in clinical decision making.
American Journal of Medicine, 121(5 SUPPL.) and Croskerry, P. (2009). A universal model of
diagnostic reasoning. Academic Medicine : Journal of the Association of American Medical
Colleges, 84(8), 1022–1028.
Type 1 decision-making approach: Heuristic, intuitive, and data-driven. Decision-
making on the basis of type 1 decision-making approach is characterized as heuristic, intuitive,
and data-driven (Croskerry & Norman, 2008; Croskerry, 2009; Hardin, 2003b). Type 1
approach assumes that humans accumulate their knowledge and experiences in a form of patterns
and store them in long-term memory (Banning, 2007; Croskerry, 2009; Hardin, 2003b; S. May,
2013). The patterned knowledge and experiences guide the user’s decision-making process
using type 1 approach: the synthesized past experiences serve as references to figure out if an
encountered situation is typical to the user. If the situation is recognized as typical, the user
11
proceeds the type 1 decision-making approach. In medical areas, the type 1 approach is referred
to as pattern recognition (S. May, 2013), forward reasoning (Hardin, 2003b), or intuitive-
humanist model (Benner 1982, 1984; Young, 1987).
Type 1 approach works quickly (McKenzie, 2014), because it works on the basis of
intuition. Intuition refers to immediate understandings of something without a rationale (Benner
& Tanner, 1987; Schrader & Fischer, 1987). Type 1 users, thus, do not require deliberate efforts
to make a decision (McKenzie, 2014). They intuitively recognize the typicality of the case using
their prior knowledge or experiences and then make an appropriate decision (Evans, 2006;
Klaczynski & Lavallee 2005; Stanovich, 1999). Because this decision-making approach asks
users to utilize their past experiences, this approach is a method that successful experienced
individuals frequently use (Banning, 2007; Hardin, 2003b).
However, the type 1 process may lead to many errors (McKenzie, 2014). Because the
type 1 process works on the basis of an individual’s past experiences, the quality of analysis for
decision-making depends on the user’s experiences (Banning, 2007; King & MacLeod, 2002).
For example, if the individual has not accumulated past experiences, type 1 reasoning may be
restricted. If the individual incorrectly refers to the past experiences, a decision may lead to
unsatisfactory results. Also, because type 1 decisions depend on the user’s intuition, the
accuracy of type 1 decisions may be affected by the users’ overconfidence or other emotional
influences (Banning, 2007; Croskerry & Norman, 2008; Croskerry, 2009; McKenzie, 2014;
Smith et al., 2008).
Type 2 decision-making: Systematic and analytic. Type 2 decision-making is systematic
or analytic (Croskerry, 2009). Whereas the type 1 process works on the basis of intuition
(McKenzie, 2014), the type 2 process involves that decision makers’ thinking processes follow
12
rational logic based on their knowledge (Banning, 2007; Cockcroft, 2007; Hardin, 2003b). In
medical areas, the type 2 process is referred to as hypothetico-deductive reasoning (Harasym,
1997; Hardin, 2003a), backward reasoning (Hardin, 2003; McKenzie, 2014), or information-
processing model (Joseph & Patel, 1990).
Decision-making on a basis of the type 2 approach involves several stages: cue
recognition and interpretation, hypothesis generation, hypothesis evaluation, and selection of the
leading hypothesis (Barrows & Tamblyn, 1980; Cockcroft, 2007). All individuals, regardless of
their experiences, can use this type of decision-making, because this process can be applied to all
types of problems (Cockcroft, 2007; Hardin, 2003b). Experienced individuals, in particular, tend
to use type 1 processes when a case is familiar and use type 2 processes when a case is not
typical.
Type 2 decision-making process leads to less errors than type 1 processes (McKenzie,
2014). Because the process requires a user’s systematic and logical thinking, the result may be
typically free from overconfidence or other emotional influences (McKenzie, 2014).
The type 2 process, however, takes slow and requires more efforts than Type 1 (Hardin,
2003b; McKenzie, 2014). Type 2 reasoning is limited by working memory capacity and, if
problems require much information to process, working memory will be overloaded (Evans,
2003; Hardin, 2003b; Slovic, Finucane, Peters, & MacGregor, 2004). Also, the type 2 process is
dependent on the information available to construct and guide formalized decision-making
(McKenzie, 2014). Thus, the type 2 process would not produce quality decision-making if either
the quantity or the quality or both of information is limited (Banning, 2007; Harbison, 1991).
As mentioned above, the dual process theory assumes that humans use both type 1 and
type 2 processes to make a decision. For example, in clinical settings, experienced doctors make
13
intuitive decisions on a basis of their patterned knowledge. The knowledge has been
scientifically and analytically derived from their extensive past experiences. The decisions are
evaluated and justified by applying their scientific knowledge (Hammond, 1996).
Naturalistic decision-making theory. Naturalistic decision-making theory was evolved
in 1989 from the reflection on the previous decision-making studies, which did not fully describe
decision-making processes in the real world (Collyer & Malecki, 1998; Klein & Klinger, 1991;
Klein, 1993, 2008; Lipshitz, Klein, Orasanu, & Salas, 2001; Lipshitz, 1993; Orasanu &
Connolly, 1993). Previous decision-making theories, commonly referred to as classical decision-
making theories, identified optimal ways of making decisions, and researchers suggested that
people should make a decision using the optimal ways (Collyer & Malecki, 1998; Klein &
Klinger, 1991; Klein, 1993, 2008; Lipshitz et al., 2001; Lipshitz, 1993; Orasanu & Connolly,
1993).
The optimal ways of decision-making begin with hypotheses. Decision makers derive
multiple hypotheses from statistical probabilities (Klein, 2007). Decision makers are encouraged
to generate multiple options, identify criteria for evaluating them, rate each option on multiple
criteria using analytical methods, and seek the best option (Klein & Klinger, 1991; Orasanu &
Connolly, 1993) .
However, these optimal ways of making decisions may be not feasible to apply the
findings to many of the real situations (Klein & Klinger, 1991; Klein, 1993, 2008; Lipshitz et al.,
2001; Lipshitz, 1993; Orasanu & Connolly, 1993). The researchers did not account for the
expertise of the decision maker (Orasanu & Connolly, 1993), the task in which a decision-
making resides (Orasanu & Connolly, 1993) and other situational factors, such as time pressure
(Klein & Klinger, 1991). For example, experienced decision makers have a single leading
14
hypothesis in their mind, rather than having multiple hypotheses, and evaluate if the hypothesis
is correct. Also, people often encounter a situation that needs to make a decision under time
pressure, but exploring multiple options and applying evaluation criteria to each option takes too
long. Thus, the previous decision-making models are most useful in well-structured settings in
which involved problem variables are controllable (Klein, 2008). Naturalistic decision-making
researchers argue that training tools or methods based on the formal standards may not
effectively elevate the quality of decisions made or be adopted in real field settings (Klein, 2008;
Yates, Veinott, & Patalano, 2003).
Naturalistic decision-making, derived from the reflection on the previous decision-
making studies, is an attempt to understand how people actually make decisions in complex real-
world settings (Klein & Klinger, 1991; Klein, 1993, 2008; Lipshitz et al., 2001; Lipshitz, 1993;
Orasanu & Connolly, 1993). The naturalistic decision-making community believes that,
although classical decision-making theory considers decision-making is choosing among known
alternatives, real-world decision-making can be best investigated by a naturalistic approach
(Patel et al., 2005). According to the community, decision makers in real world rely on
heuristics, as opposed to algorithmic strategies, to make a decision under complex conditions,
characterized by limited time, uncertainty, high stakes, vague goals, and unstable conditions
(Klein, 2008).
SOAP (Subjective and Objective observation, Assessment, and Plan). To support
veterinary students’ clinical decision-making process, SOAP, a thinking model as well as a
clinical tool, has been used for many years in a number of colleges of veterinary medicine (S.
May, 2013; Riegger, 2011). The SOAP stands for subjective observation (S), objective
observation (O), assessment (A), and plan (P) (Riegger, 2011). This format well-represents the
15
process of clinical decision-making, including collecting clinical data and framing data-driven
forward reasoning.
The first two parts represent the phase of data collection (Cameron & Turtle-song, 2002).
Subjective observation asks students to collect information from the perspective of a patient or
owners through history taking (Cameron & Turtle-song, 2002). For example, the patient’s
feelings (e.g., responsive, depressed, or lethargic) and the owner’s concerns and thoughts can be
collected (Cameron & Turtle-song, 2002; Riegger, 2011).
Objective observation is collecting factual and measurable information which is matter-
of-fact and dry without opinions (Cameron & Turtle-song, 2002; Riegger, 2011). There are two
types of objective observation: the veterinarian’s observations and outside written-materials
(Cameron & Turtle-song, 2002). Veterinarian observations include any physical findings that
the veterinarian witnesses. The findings should be precisely and descriptively stated. Outside
written materials include reports obtained from other veterinarians, physical examination results,
or medical records (Cameron & Turtle-song, 2002; Riegger, 2011).
Through subjective and objective observations, veterinarians do an assessment. During
this assessment phase, a veterinarian is asked to analyze and synthesize the data acquired from
the subjective and objective observations (Cameron & Turtle-song, 2002) and list differential
diagnoses and prognosis (Riegger, 2011). The goal of assessment is to identify potential
problems, which will lead to a diagnosis (Riegger, 2011).
Plan, the last stage of SOAP, asks veterinarians to write a plan of therapeutic and
behavioral actions to solve the patient’s problem(s) based on the subjective and objective data,
and assessment (Riegger, 2011). The plan includes (a) any additional diagnostic plans to further
16
define the problem, (b) treatment plans to address the problem, and (c) plans for communicating
with owners (Riegger, 2011).
Knowledge Application
In order to make a good clinical decision, a veterinarian must possess adequate
knowledge and appropriately apply the knowledge to a given context. Educators often place an
emphasis on the amount or depth of medical knowledge, but more important for clinical
decision-making than the quantity and the quality of knowledge is how the knowledge is used for
decision-making in real context (Cutrer, Sullivan, & Fleming, 2013). When teaching veterinary
learners, instructors should ensure that knowledge the learners have acquired at school is easily
accessible and useful for clinical decision-making in real context (Cutrer et al., 2013). In this
section, veterinary students’ typical difficulty in knowledge application and teaching methods to
enhance the application are discussed.
Veterinary Education and Knowledge Application
Historically, teacher-centered lecture methods were dominant in many higher education
courses, including veterinary education (Whitney, Herron, & Weeks, 1993). The explosion of
knowledge and the size of classes might be the primary driving forces for lectures, especially for
large classes (Fletcher, Hooper, & Schoenfeld-Tacher, 2015). Lectures are one of the most
efficient and cost-effective teaching methods to cover a large body of knowledge to many people
(Campanella & Lygo-Baker, 2014). However, one of the limitations of lectures is that they do
not cover knowledge at a deeper level. For this reason, research that has reported the lack of
knowledge application skills of veterinary students questions the value of knowledge transmitted
by lectures.
17
Case-based learning has been adopted in many veterinary colleges in order to enhance
students’ abilities to apply learned knowledge to clinical settings (Fletcher et al., 2015). Case-
based learning refers to a pedagogical method that enables students to interpret, reflect on, and
apply their own or someone else’s experiences while participating in real or authentic situations
(Ertmer & Russell, 1995; Kolodner, Owensby, & Guzdial, 2004).
Advocates of case-based learning have credited case-based learning for shifting the focus
of learning from memorization to application by bridging the gap between theory and practice
(Ertmer & Russell, 1995; Flynn & Klein, 2001; Hansen, Ferguson, Sipe, & Sorosky, 2005;
Thistlethwaite et al., 2012; Williams, 1992). Most knowledge learned at schools is abstract and
decontextualized (J. S. Brown et al., 1989). Unsituated knowledge without specific context
where students can connect the dots between knowledge and context often leads the students to
have difficulty in utilizing their learned knowledge for any critical or deep purposes (J. S. Brown
et al., 1989; Gee, 1997). In other words, “what is learned” (p. 32) is separated from “how it is
learned and used” (p. 32) (J. S. Brown et al., 1989). To overcome the separation, it is important
for novices to be enculturated by experiencing real or similar communities and cultures (J. S.
Brown et al., 1989). Case-based learning encourages students to integrate their knowledge into
the context of real or realistic scenarios (Thistlethwaite et al., 2012).
Also, advocates of case-based learning have identified power of stories to explain the
effectiveness of case-based learning. Stories are the “means [by] which human beings give
meaning to their experience of temporality and personal actions” (Polkinghorne, 1988, p. 11).
Stories require less cognitive effort (Jonassen & Hernandez-Serrano, 2002; Williams, 1992),
because their structure has a similar format to our lives, which makes it easier for students to
generate a mental model of the situation (Williams, 1992). Stories were initially advocated due
18
to their entertainment value (Jonassen & Hernandez-Serrano, 2002) and now have begun to play
a more important role in social science studies (Jonassen & Hernandez-Serrano, 2002). In
particular, stories are helpful to train novices, because they provide vicarious experiences to the
novices who lack direct experiences (Jonassen & Hernandez-Serrano, 2002).
Case-Based Learning and Knowledge Application in Veterinary Education
Many researchers and educators have reported the effects of case-based learning in
veterinary education. Case-based learning in veterinary education is effective in facilitating
knowledge integration, development of expertise, clinical reasoning, problem solving, and
decision-making.
First, case-based learning facilitates verification, application, and integration of
understanding of core concepts to real-life situations (Hansen et al., 2005; Thistlethwaite et al.,
2012). For example, Sharkey, Overmann, and Flash (2007) provided second-year students with
case-based writing assignments, including a clinical history, physical examination findings, and
laboratory data. The results indicated that the students reported increased confidence in
understanding of the content and abilities to apply the knowledge. Students in a study by Malher
et al. (2009) also reported increased confidence in assimilating and integrating the concepts they
previously learned. Grauer, Forrester, Shuman & Sanderson (2008) compared traditional
lecture-based learning and case-based learning for third-year veterinary students. They tested
three differing knowledge: knowledge/factual, knowledge with application, and application and
analysis. The results showed that students in case-based learning group outperformed their peers
in the traditional lecture group in the application and analysis. Monaghan and Yew (2002)
observed that third-year veterinary students who participated in a case-based parasitology
19
laboratory experienced enhanced understanding of the clinical concepts and valuable clinical
insights.
Second, case-based learning supports the development of expertise (Cannon-Bowers &
Bell, 1997). It is clear from research evidence that experts make clinical decisions based on their
expertise (e.g., Recognition-Primed Decision model, pattern recognition). The ideal situation to
develop the expertise would be having students experience as many real practices as possible
(Ladyshewsky & Jones, 2008), like experts have done through their experiences over time.
However, real clinical settings are characterized by their complexity, uncertainty, high-risk, and
time pressure, making it difficult to train students in real practices (Terry & Higgs, 1993). In
other words, time pressure and personal and professional expectations for satisfactory results can
be demanding for learners to take time out to engage in mindful practice through persistence,
struggle, and self-reflection (Kassirer, 2010).
Third, case-based learning is effective in promoting students’ higher order thinking, such
as clinical reasoning, critical thinking, problem solving, and decision-making (Thistlethwaite et
al., 2012). Case-based learning requires learners to utilize their critical thinking skills to
approach a given case or problem situation. To elaborate, they need to analyze the situation,
identify possible solutions to solve the situation, and suggest the best or better solutions (Flynn &
Klein, 2001). In a study by Mahler, Bareille, Noordhuizen, and Seegers (2009), for example, a
case-based learning approach was employed to help undergraduate veterinary students learn
about dairy herd health consultancy. The students were asked to identify problems and
recommend action plans for implementation. The participants showed positive responses about
the case-based problem in that it was effective in developing their problem-solving skills in the
fields of dairy herd health management. Patterson (2006) examined second-year veterinary
20
students’ confidence in performing clinical reasoning skills through practicing in small-group
case discussion. The researcher found significant increases in students’ self-confidence in
clinical reasoning skills.
In addition, case-based learning seems to be beneficial in increasing students’ interests in
the topic (Koh et al., 1995; Hansen et al., 2005) and competence in performing tasks (Patterson,
2006). These benefits seem to stem from the features of case-based learning which puts learners
into situations that encourage them to actively participate in the process of problem solving,
including hypotheses generation, data collection, data interpretation, and solution generation
(Kolodner, Hmelo, & Narayanan, 1996).
Educational Implications
Previous research on case-based learning and/or clinical decision-making has suggested
several educational implications that could enhance the clinical decision-making skills: providing
an authentic practice experience, providing an entire cycle of decision-making, providing expert
decision-making processes, and encouraging discussions.
Providing an authentic practice experience. One of factors that prevent novices from
applying knowledge or skills to real contexts is a lack of realistic activities, contexts, and
cultures (J. S. Brown et al., 1989). In order to train novices to deal with the complex, uncertain
real problems, they should be exposed to stories, cases, and problems generated in the real
settings (J. S. Brown et al., 1989; Jonassen & Hernandez-Serrano, 2002).
Also, the effectiveness of case-based learning is dependent on the quality of the cases.
One of the important considerations for creating educationally valuable cases is whether the case
includes authentic problems with real-world challenges (Choi et al., 2013). By providing real or
realistic cases, students can be situated in contexts in which their learned knowledge can be used.
21
In other words, what they have learned can be closely connected with how the knowledge is used
(J. S. Brown et al., 1989).
Providing an entire decision-making cycle. Case-based learning for enhancing
decision-making skills should reflect the decision-making process in practices (Thistlethwaite et
al., 2012). Williams (1992) made suggestions to include cases that help novices acquire a broad
range of problem-solving skills. By being provided the entire clinical decision-making cycle of
situation assessment, data interpretation, and treatment plan, novices are expected to learn how to
evaluate patient’s problem or clinical data, perform a diagnosis, and provide treatment.
Providing expert decision-making processes. According to the dual process theory and
other research on clinical decision-making, most experienced decision makers use of both
heuristic (i.e., type 1 in dual process theory) and analytical (i.e., type 2 in dual process theory)
cognitive processes to make a decision in real settings (Benner et al., 1992, 1996; Crow & Spicer,
1995; Easen & Wilcockson, 1996; Grobe et al., 1991; Lauri, 1992; Thiele et al., 1991). On top
of that, several researchers (e.g., Norman et al., 1994; Patel et al., 2005; Rikers, Loyens, &
Schmidt, 2004) have found that the heuristic approach, or data-driven approach, better supports
the development of clinical reasoning skills. To help frame novices’ thinking processes, thus, it
is important to provide a decision-making model based on a data-driven approach, such as
naturalistic decision-making models or SOAP tool, rather than the hypothesis-driven approach.
Encouraging discussions. Many theoretical and empirical studies have supported
benefits of discussions in case-based methods (Thurman, Volet, & Bolton, 2009). Case-based
learning does not always involve discussions with colleagues and experts, but collaborative
learning with them can be more effective and efficient than individual learning (Choo, Rotgans,
Yew, & Schmidt, 2011; Flynn & Klein, 2001; Sato, 2013; Thistlethwaite et al., 2012; Thurman
22
et al., 2009; Uribe, Klein, & Sullivan, 2003). In terms of efficiency, for example, learners can
cover more knowledge in a shorter amount of time than they do alone (Flynn & Klein, 2001). In
terms of effectiveness, discussions can encourage learners to consider learning content more
deeply than they do alone (Mann, Gordon, & MacLeod, 2009).
Discussions, especially with peers, allow learners to challenge others’ ideas more freely
than having discussions with experts (Mann et al., 2009). Schon (1987) emphasized the
importance of an open climate wherein people can freely exchange information, opinions and
feedback. For example, Droge and Spreng (1996) compared instructor-driven and learner-driven
discussions and found that learner-driven discussions were beneficial in terms of efficiency (e.g.,
use of time) and effectiveness (e.g., satisfaction, achievement educational goals, and
competencies).
In addition to this, group learning can enhance understanding of related topics. Levin
(1995) compared the effectiveness of two discussions: one among experienced teachers and the
other among inexperienced teachers. The results indicated that both groups of teachers
benefitted from discussions. In particular, discussions were beneficial in experienced teachers’
reflection and inexperienced teachers’ enhanced understanding of the topic. Also, researchers in
veterinary or medical education (e.g., Volet, Summers, & Thurman, 2009) have observed that
students who participated in collaborative learning activities had a better understanding of
medical knowledge.
Reflection
Research on the characteristics of expert clinicians’ reasoning proves that experts
frequently monitor and manage their cognitive processes to make better decisions (Higgs &
Jones, 2008; Wojcikowski & Brownie, 2013). In other words, the research shows that experts
23
put more effort into reflection. For example, experienced physicians spent more time on the
verifications of their diagnosis than less experienced physicians did (Patel & Groen, 1991). Also,
experts kept revisiting and refining their initial representations throughout the practice (Chi,
Glaser, & Rees, 1982). What is reflection? What is the role of reflection in decision-making
actions?
Definitions of Reflection
Many researchers have defined reflection based on the characteristics of reflection. Some
researchers have indicated that reflection is a deliberate thinking. Dewey defines reflection as
“active, persistent, and careful consideration of any belief or supposed form of knowledge in the
light of the grounds that support it and the further conclusions to which it tends” (p. 9). He adds
that a state of doubt or uncertainty, such as a complex situation, errors or unexpected
results(Epstein, 1999), encourages individuals to seek possible explanations or solutions of the
unstable state, which results in provoking reflective thinking. Mann et al. (2009) also agree that
awareness of a need can lead to reflection.
Other definitions of reflection have highlighted the role of reflection, especially in
uncertain situations, such as problem solving and/or decision-making. For example, Moon
(1999) defines reflection as “form of mental processing with a purpose and/or anticipated
outcome that is applied to relatively complex or unstructured ideas for which there is not an
obvious solution” (p. 23), and Van Manen (1991) defines reflection as “the connotation of
deliberation, making choices, coming to decisions about alternative courses of action” (p. 511).
Schon also regards reflection as an interaction that occurs between the problem solver and a
surrounding problematic situation.
24
Some definitions have emphasized that reflection can lead to an improved outcome. For
example, Boud et al. (1985) stated that reflection is “a generic term for those intellectual and
affective activities in which individuals engage to explore their experiences in order to lead to a
new understanding and appreciation” (p. 19).
To summarize, reflection refers to a deliberate thinking about experience, such as one’s
own thinking, goals or content. As a result of reflection, individuals may expect an improved
quality of the outcome.
Benefits of Reflection
Benefits of reflection in general learning. Historically, researchers and educators have
emphasized the importance of reflection, especially in learning (e.g., Bloom, 1956; Boud &
Walker, 1998; Brown, Bransford, Ferrara, & Campione, 1983). The importance of reflection for
the learner stems from two benefits that emerge during the learning process: engaging
individuals in deeper learning and lessening the gaps between theory and practice.
Firstly, reflection can enhance an individual’s abilities to associate and integrate
information, which in turn can lead to deeper learning (Mann et al., 2009). Reflection may not
directly elicit better understanding of knowledge, but it can help students focus more on the
structure of knowledge instead of its superficial features (Davis & Linn, 2000). In paying more
attention to the structure of knowledge, student are then able to engage in knowledge integration
processes like creating links between information, integrating the newly learned knowledge into
their prior knowledge, and restructuring as well as expanding their knowledge system (Davis &
Linn, 2000; Davis, 2003). Many empirical studies provide support to the hypothesis that
reflection can lead to better understanding of knowledge, such as more precise and sophisticated
knowledge (e.g., Land & Zembal-Saul, 2003) and better knowledge integration (e.g., Davis &
25
Linn, 2000; Davis, 2003) in learning. Specifically within the fields of veterinary and human
medicine, the act and practice of reflection is presumed to allow clinical knowledge to replaced
biomedical knowledge (Mamede & Schmidt, 2004).
Additionally, productive reflection is thought to lessen the gaps between theories and
practices (Mann et al., 2009). Given that theoretical knowledge is abstract and lacks specific
contexts, it could be difficult for novices to immediately make connections between theory and
practice (A. L. Brown, 1987). Reflection, here, helps students dissect the structure of the
knowledge, become aware of their own or someone else’s relevant experiences in learning
processes, and make relationships between the knowledge and the relevant experiences based on
what they know (Moon, 2004; van den Boom, Paas, & van Merriënboer, 2007).
Benefits of reflection in clinical decision-making. Research has shown that reflection
plays a key role in leading successful clinical decision-makings in the fields of general medicine
(Epstein, 1999; Mamede & Schmidt, 2004, 2005), nursing (Murphy, 2004; Rashotte &
Carnevale, 2004), physical therapy (Atkinson & Nixon-Cave, 2011), and other health professions
(Mann et al., 2009). However, researchers still search for why reflection is so important in
making clinical decisions.
Some researchers have found that reflection may be rigorously required in uncertain,
complex, and ill-defined areas (Schon, 1983), and clinical decision-making is one of the ill-
defined areas (Mamede & Schmidt, 2004; Maudsley & Strivens, 2000). Unsurprisingly, research
has also found that performing a task in ill-defined settings is challenging, because there is not a
single absolute solution (Jonassen, 1997). Even if there is an appropriate and proven solution, it
may not work for another similar problem. On top of that, available resources may be
ambiguous, or sometimes its validity can be suspect (Orasanu & Connolly, 1993). Thus, in these
26
ill-defined settings, reflective processes to plan, monitor, and evaluate the course of actions seem
to be more rigorously required (Bransford, Brown, & Cocking, 2000; Lin, Hmelo, & Kinzer,
1999; Shin et al., 2003). Additionally, during clinical decision-making processes, even experts
can protect themselves against errors if they reflect upon their decision-making process (Higgs &
Jones, 2008). A good decision maker may reflect on the underlying norms, strategies, and
theories implicit in the processes (Higgs & Jones, 2008).
Moreover, reflection can serve as a new learning experience (Schon, 1983, 1988).
Through reflection, a clinician critically reanalyzes the situation and can further identify
underlying concepts. The clinician refines his past experiences or constructs a new experience
for future clinical decision-making. Likewise, novice decision makers who are lacking in
reflective thinking skills may not be able to learn as much as experienced decision makers could
(Ladyshewsky & Jones, 2008).
These findings indicate that it is important to encourage students to reflect upon their
thinking processes while making a clinical decision, which may result in an improved quality of
their decision (Croskerry & Nimmo, 2011; Epstein, 1999; Jones, 1992; Mamede & Schmidt,
2004, 2005). Recently, practitioners learned the importance of reflection in their actual practice
and have incorporated reflection into educational courses or programs ranging from initial
training to continuing education. As a result, the concept of reflection can be found in various
curriculums, including teaching (e.g. Zeichner & Liston, 1987), social work (e.g. Gould &
Taylor, 1996), general medicine (Epstein, 1999; Mamede & Schmidt, 2004, 2005), nursing
(Murphy, 2004; Rashotte & Carnevale, 2004), physical therapy (Atkinson & Nixon-Cave, 2011),
veterinary education (Khosa, Volet, & Bolton, 2014) and other health professions (Mann et al.,
27
2009) where students are required to closely integrate academic studies and field experiences
(Boud & Walker, 1998).
Scaffolded Reflection
Students often have difficulty engaging in reflective thinking for many reasons. One
such reason is that students rarely learn the importance and benefits of reflection (Boud &
Walker, 1998). On top of that, leaving time for reflection does not always guarantee productive
reflection (Boud & Walker, 1998), because students can fail to reflect mindfully on their
knowledge or a course of action (Woodward, 1998). Without appropriate guidance and
directions, students may spend the time for reflection by doing something else other than
reflection (van den Boom et al., 2007). Thus, scaffolding has been identified as a promising
method to guide students’ reflection in many studies (e.g., Davis & Linn, 2000; Kassirer, 2010;
Khosa et al., 2014; Land & Zembal-Saul, 2003; Quintana, Zhang, & Krajcik, 2005; Yang, 2011).
The term scaffolding originally refers to a physical support used for the construction of a
building, which is later removed when the building's construction is complete. Scaffolding in the
context of learning refers to an aid provided by experts or more knowledgeable people to assist
novices or students to complete or perform a task (Collins et al., 1991, 1987; Wood, Bruner, &
Ross, 1976).
The key concept of scaffolding is that an external support or aid is provided to enable a
learner to accomplish a certain task that would otherwise be out of reach (Wood et al., 1976).
This concept is associated with Vygotsky’s Zone of Proximal Development (ZPD), which
Vygotsky (1978) defines as the gap of competence. To elaborate, this gap is between the state
where the person performs the task by himself and the state where the same person performs a
task with an external aid, which refers to scaffolding. Therefore, the gap of competence refers to
28
the levels of competence between a person doing a task with and without scaffolding. Thus, in
order to scaffold a student's performance, learning tasks should be situated in the ZPD (Dennen,
2004; Vygotsky, 1978). In the ZPD, students are able to participate in activities beyond their
own performance level, which enables students to utilize cultural tools to adapt themselves to the
specific activity at hand (Dennen, 2004).
Scaffolding a person’s performance assumes that there are shared understandings
between an agent who provides an aid and an aided person (Dennen, 2004). Although agents
and learners may have differing understanding about the task, they eventually share a common
understanding through collaborative interaction. This shared understanding is called
intersubjectivity. Without intersubjectivity, students may experience learning conflicts, low
participation levels, low engagement, or unexpected learning outcomes (Dennen, 2004).
Scaffolding also assumes that people actively construct their own knowledge (Dennen,
2004). Agents provide an external aid for students to facilitate their learning, but that does not
mean that agents are the center of cognitive activity. Learners’ participation has to be key to the
activity in pursuit of the learning goals (Dennen, 2004; Stone, 1993). It should be noted that in
due time, scaffolds must be removed so that learners can ultimately perform a task alone (Sherin
et al., 2004).
Educational Implications
In order to support students’ reflection in the process of decision-making, instructors
need to identify appropriate methods and tools to intentionally encourage students to make their
thinking visible (Boud & Walker, 1998; Linn, 2000). Based on studies on reflection, several
educational implications to facilitate students’ reflection during clinical decision-making were
identified.
29
Having students take time to reflect. Although reflection is important in learning,
students rarely realize the importance of reflection and thus, fail to engage in reflective thinking
at a deeper level (Boud & Walker, 1998). As indicated previously, reflection is a purposeful
action (Mann et al., 2009). Individuals should take time to reflect upon the actions they selected
and upon the situations they encountered. A five-factor structure of reflective practice found by
Mamede and Schmidt (2004) supports the importance of reflection in clinical practice: deliberate
induction, one of the five factors, refers to taking time to reflect upon an unfamiliar problem.
Likewise, it is important to have or remind novices of taking their time to reflect on their actions
and situations surrounding them.
Providing an opportunity to be exposed to expert performance. It may be important
for novices to receive opportunities to learn how experts use domain knowledge and strategies as
well as make a clinical decision when encountering authentic, uncertain, and ill-defined cases (J.
S. Brown et al., 1989; Williams, 1992). Experts may demonstrate their internal cognitive
process by physically carrying out a task or verbalizing their performance. Verbalization, in
particular, provides an opportunity for students to explicitly observe an expert’s internal
cognitive process as they engage in their decision-making process (Collins et al., 1987; Pedersen
& Liu, 2002). Thus, the expert’s demonstration enables students to build a task-related problem
space in a quicker and more accurate way (Collins et al., 1991, 1987; Williams, 1992), and,
subsequently, they are expected to internalize those behaviors and strategies demonstrated by
experts at the individual level (Collins et al., 1991, 1987).
Being exposed to experts’ decision-making process is expected to significantly benefit
students, since most novices or students approach problem solving by referring to known
examples or developing abstract declarative rules that guide their problem solving (Anderson,
30
1987; Pirolli & Anderson, 1985). Thus, expert commentary would work for them as an excellent
reference or models of behavior for their performance.
Providing an opportunity to give feedback to others. One of the promising methods to
promote learners’ reflection upon their thinking, their actions or inactions, as well as the contexts
of various situations encourages them to explore other people’s ideas while engaging in
discussions with peers and/or instructors (Boud & Walker, 1998; Ladyshewsky & Jones, 2008;
Rogoff, 1990; van den Boom et al., 2007), which can help the students revisit and reassess their
own actions and thinking (Land & Zembal-Saul, 2003; O’Malley & Scanlon, 1990).
Feedback among peers, in particular, is recommended as an effective means to promote
reflective processes for two reasons. Firstly, peer feedback facilitates cognitive growth
(Ladyshewsky & Jones, 2008; Rogoff, 1990) and influential interaction can occur between
partners of similar status (Rogoff, 1990). Peers who share joint problem-solving activities would
feel freer to examine the logic of arguments than peers who only interact with adults or experts
(Ladyshewsky & Jones, 2008; May & Newman, 1980; Rogoff, 1990; Terry & Higgs, 1993).
Furthermore, the ability to examine the logic of arguments leads to more effective problem
solving (Ladyshewsky & Jones, 2008; May & Newman, 1980; Rogoff, 1990; Terry & Higgs,
1993). The second reason is that peer feedback is more efficient in terms of costs as compared to
having supervised small-group discussions or bedside teaching (Borleffs, Custers, van Gijn, &
ten Cate, 2003; Ladyshewsky & Jones, 2008). From a practical point of view, time pressures and
workloads may be one of the factors that make it difficult for supervisors to explore students’
clinical reasoning in action (Ladyshewsky & Jones, 2008). When peers serve as cognitive
facilitators for one another, a teacher-to-student ratio does not matter in peer interaction activity.
31
Providing multiple practice experiences. Research suggests that reflective thinking can
be stimulated by complex problems (Epstein, 1999; Mamede & Schmidt, 2005). To elaborate,
reflective thinking skills can be further enhanced when individuals continuously revisit similar
complex problems, due to the fact that one’s reflective thinking ability can be framed and
promoted as expertise develops (Mamede & Schmidt, 2005; Mann et al., 2009). For example,
one study found that learners reported that their reflective thinking ability had improved after
participating in clinical practice experiences (Hallett, 1997). Another study found further
support for the importance of clinical practice experiences (Song, Grabowski, Koszalka, &
Harkness, 2006). According to this second study, college-level learners identified ill-structured
tasks as one of the important elements for their reflective thinking.
Conceptual Framework
Based on the implications from the literature review, several instructional designs to
develop students’ clinical decision-making skills were identified. These instructional designs
can be divided into two purposes: to enhance students’ knowledge application and to promote
their reflection as presented in Figure 2-1. Case-based teaching methods were employed to
enhance veterinary students’ knowledge application, and scaffolded revision activities were
employed to promote their reflection. Specific instructional designs employed for the case-based
teaching methods and scaffolded revision activities are briefly described in the following section.
32
Figure 2-1
Conceptual framework for this study to enhance veterinary students’ clinical decision-making
skills
Case-based online learning module. To enhance veterinary students’ knowledge
application, case-based teaching methods were employed. A case-based online learning
environment for veterinary students was developed by adapting from Choi’s case-based learning
model (Choi et al., 2013, 2009; Choi & Lee, 2009; Choi, 2009) and findings from other related
research that have been described in the previous sections. To elaborate, the current case-based
module was designed based on the four main educational implications: (1) providing an
authentic experience; (2) providing an entire decision-making cycle; (3) providing expert
decision-making processes; and (4) encouraging discussions.
33
In order to provide an authentic experience, the case-based online learning module
utilized realistic clinical cases. In particular, the clinical cases were equipped with real-world
challenges (Choi et al., 2013) based on the understanding of the educational objectives of a
course in which the module was implemented (Hansen et al., 2005). More specific descriptions
about each learning element with actual examples will be discussed in the next section, Methods.
In order to provide an entire decision-making cycle which could accurately reflect the
reality of a veterinarian’s practice, the case was organized according to critical decision points of
collecting data, announcing a diagnosis, performing a surgery, prescribing medications, and
identifying a follow-up plan.
In order to provide expert decision-making processes, a decision-making model based on
the expert decision-making process was provided. The suggested decision-making model
consisted of four steps: identifying key information, assessing the case, prioritizing issues and
objectives, and planning a course of action. Also, in order to better support novices’ decision-
making processes, critical thinking prompts were added to frame and guide them in what to think
during decision-making processes.
In order to encourage discussion, peer group discussions were utilized. The students in
the discussion groups were encouraged to share their decision-making processes and outcomes
with their peers and then provide feedback to each other.
Scaffolded revision activities. In order to promote students’ reflective thinking during
the decision-making process, scaffolded revision activities were employed. The scaffolded
revision activities were designed based on the four main educational implications identified
through the literature review: (1) having students take time to reflect; (2) providing an
opportunity to be exposed to expert performance; (3) providing an opportunity to give feedback
34
to others; and (4) providing multiple practice experiences. More specific descriptions about each
learning element with actual examples will be discussed in the next section, Methods.
First, in order to have students take time to reflect upon their own practices, they were
asked to focus on two key activities: (a) make their own decision, and (b) revisit and revise their
decision. Since novices rarely engage in reflective thinking which is a purposeful action, it is
important to facilitate novices to take time to reflect on their thinking and actions as well as think
upon how to improve the quality of the outcome (Mamede & Schmidt, 2004).
In order to provide an opportunity to be exposed to expert performance and to give
feedback to others, the scaffolded revision activities encouraged novices to explore others’ ideas
with prompts to revisit and reassess their own actions and thinking (Land & Zembal-Saul, 2003).
To elaborate, expert commentary demonstrated how experts approach a clinical case and make a
decision, and peer feedback provided students with opportunities to exchange feedback with
their peers.
Lastly, in order to provide multiple practice experiences, five decision points were
designed within a series of complex problems, which were expected to stimulate novices’
reflective thinking (Mamede & Schmidt, 2005). Given that one’s reflective thinking ability can
be framed and enhanced as their expertise develops, it is important to provide novices with
multiple practice experiences (Mamede & Schmidt, 2005).
35
CHAPTER 3
METHODS
This chapter provides information about methodology employed for this study. To
elaborate, the first section provides information about participants, the course the participants
recruited from, and interventions the participants received. Next section presents research design
including research questions that this study focused on. Lastly, this chapter describes data
collection and analysis plan.
Participants
The participants for this study were chosen due to their participation in one of the third-
year core courses, specifically the Small Animal Digestive Diseases course. In total, 102 third-
year students were given the opportunity to participate in the study. During the early stages of
data collection, however, only 100 students agreed to participate in the research by filling out the
printed informed consent form.
After giving consent, the instructor of the course introduced a case-based online learning
module to the students. From there, the participants were allowed to self-select between
conditions of scaffolded revision activity: expert commentary only (EC/NP), expert commentary
as well as early peer feedback (EC/EP), and expert commentary as well as later peer feedback
(EC/LP).
Subsequently, over twenty students signed up for each peer feedback session. During the
actual data collection, however, nine students actually participated in the early peer feedback
group, while 13 students participated in the later peer feedback group. The remaining 78
36
students who did not sign up for or participate in either peer feedback groups were automatically
assigned to expert commentary only.
To balance the number of participants, 25 participants from the EC group receiving
expert commentary only were purposefully selected based on their GPA and gender. Eventually,
participants for this study were 25 students (19 female and 6 male) for the expert commentary
only group (EC/NP), nine students (8 female and 1 male) for the expert commentary with early
peer feedback (EC/EP), and 13 students (11 female and 2 male) for the expert commentary with
later peer feedback group (EC/LP) (see Table 3-1).
The average GPAs of for the EC/NP, EC/EP, and EC/LP groups were 3.52 (SD = .34), 3.
59 (SD = .30) and 3.41 (SD = .40) respectively. To test the homogeneity of the three groups,
Levene’s test was conducted. For the GPA variable, the F value for Levene’s test is 1.209 with a
p value of .308. This result indicates that there is no significant difference between the three
group’s variances.
Table 3-1
Participants’ gender, averaged GPA, and the result of Levene’s Test
Number of participants GPA Levene’s test
Female Male Total Mean SD p
EC/NPa 19 6 25 3.52 .34
.308 EC/EPb 11 2 13 3.59 .30
EC/LPc 8 1 9 3.41 .40
Overall 38 9 47 3.5 .35
Note. aEC/NP indicates the expert commentary only group.
bEC/EP indicates the expert
commentary with early peer feedback group. cEC/LP indicates the expert commentary with later
peer feedback group.
37
The Small Animal Digestive Diseases Course
Veterinary school students learn about scientific foundations in their first two years, and
then in the following two years, they are increasingly involved in clinically oriented coursework
and finally are trained in completely clinically-based situations. Since third-year students are
required to apply their medical knowledge to make clinical decisions in real clinical settings, it is
the right moment for them to train their clinical decision-making skills with case-based learning.
The course titled Small Animal Digestive Diseases was one of the required courses for
junior students to learn medical knowledge about the diagnosis and management of the medical
and surgical digestive disorders affecting dogs and cats. This course was chosen as a potential
data collection site, because objectives of the course were well aligned with the goals of the case-
based learning environment. Upon the completion of the course, students were expected to do
the following:
Describe major digestive symptoms such as anorexia, weight loss, regurgitation, and
vomiting;
Understand how to clinically and diagnostically approach these digestive problems;
Understand the role of surgery in digestive disease;
Identify digestive issues and generate a basic diagnostic and treatment plan for a clinical
case of digestive disease.
Case-Based Online Learning Module
In this study, a case-based online learning module provided a web-based context in which
students identified, explored, and reconstructed an authentic veterinary case. The case-based
online learning module was developed based on Choi’s case-based learning model (Choi et al.,
38
2013, 2009; Choi & Lee, 2009) and educational implications that have been identified in this
current study.
Sample pages of the case-based online learning module are presented in Figure 3-1 and
Figure 3-2. Figure 3-1 features the page of an initial decision-making activity with aids of a
realistic case video and critical thinking prompts. Figure 3-2 features the page of a revision
activity with an aid of expert commentary videos.
Figure 3-1
A sample screen of an initial decision-making activity in the case-based learning environment
39
Figure 3-2
A sample screen of a revision activity in the case-based learning environment
The case-based online learning module was implemented in the Small Animal Digestive
Diseases course to enhance students’ knowledge applications in digestive diseases of small
animals. To elaborate, students were expected to understand and diagnose major digestive
symptoms as well as generate basic treatment plans for a realistic case of digestive diseases upon
the completion of the course and this module. Thus, several learning elements were identified
and then utilized as follows: realistic case videos, a decision-making model based on veterinary
experts’ past experiences along with critical thinking prompts, expert commentary videos,
medical records and digital textbook. More specific information on each learning element is
presented in the following sections.
40
Case Videos
The first learning element used was the case video, because it visually and auditorily
represented a realistic clinical case (Choi et al., 2013, 2009; Choi & Lee, 2009). A team of
experienced veterinary faculty members identified a realistic clinical case based on their past
experiences. The clinical case provided in the learning module was about a 4-year-old Westie
who had vomited several times at the previous night. It seemed to be some ‘dietary indiscretion’
but the symptoms were persisting. Students, as a doctor of his, were asked to diagnose and treat
him.
For this research, the educators identified five critical points in which students need to
make a decision as presented in Figure 3-4, and the single case video was segmented into five
small case videos accordingly. Square nodes indicate medical cases in which require students to
make a decision, and rounded square nodes indicate options that students can choose from.
41
Figure 3-3
Five Decision Points in the module
42
A decision-making model with critical thinking prompts
The second learning element was made up of two integrated parts: the first part being a
decision-making model based on experts’ decision-making process and the second part being
critical thinking prompts to guide each stage of the model (Choi et al., 2013, 2009; Choi & Lee,
2009). The decision-making model was comprised of four phases, [Identifying key information]
– [Assessing the current case] – [Prioritizing issues and objectives] – [Making an immediate plan]
based on the SOAP writing, which is a thinking and decision-making model developed for
veterinary students.
To support students’ decision-making process, the experienced veterinary educators
identified and developed the critical thinking prompts corresponding to each phase of decision-
making. In the phase of Identifying key information, for example, five or more cues were listed,
and students were asked to identify two or more cues that they think more important than others.
In the phase of Assessing the current case, they were asked to describe how they would utilize
the cues identified in the previous phase. In the phase of Prioritizing issues and objectives, they
needed to set their goals for their course of action. In the last phase, Making an immediate plan,
students were asked to select one single action for their patient out of three or four actions and
justify their decision. The prompts were specifically designed for each Decision Point and asked
the students to verbalize their ideas and opinions. The critical thinking prompts at Decision
Point 1 are presented in Table 3-2.
43
Table 3-2
The question prompts embedded at Decision Point 1 in the learning module
Decision-making
Process
Required Activity Sample question prompts
Identifying key
information
Watching an authentic
case video and
analyzing important
cues
[Multiple-choice question]
From the list provided, please check those items
that you are most heavily considering in your
decision-making process.
1. Known history of eating garbage
2. Physical exam findings
3. Inflammatory leukogram
4. Hemoconcentration (high Hct / TP)
5. Slightly low chloride
6. Presence of focal ileus on radiographs
7. Presence of fluid-filled intestine on
radiographs
Current case
assessment
Identifying and
justifying the problem
situation using the key
information in the
previous step
[Open-ended question]
Please describe how you utilized the cues
selected above in your thought process about the
next action step for Doug.
Prioritizing issues
and objectives
Prioritizing involved
factors that can affect
the final decision
[Multiple-choice question]
You will be asked to make a decision about the
next diagnostic and/or therapeutic step for Doug.
Before you make that decision, you must
consider the needs, goals and objectives for
Doug and his owners. Please write out your
thoughts on each of these aspects (goals,
questions and/or needs), prior to making a
decision.
44
Decision-making
Process
Required Activity Sample question prompts
Immediate plan Planning and providing
justifications a course of
action based on their
own analysis
[Multiple-choice and Essay questions]
Based on your problem identification and
assessment, please select the single action you
would take next for Doug.
1. Admit Doug to your clinic for supportive
care, in the form of IV fluids,
antiemetics, careful introduction of a
bland diet, and close monitoring.
2. Recommend surgical exploratory of
Doug
3. Investigate further by performing
abdominal ultrasound, abdominocentesis,
and possibly an upper GI barium contrast
study.
Please explain your choice.
Expert commentary videos
The four veterinary faculty members narrated how they would have accessed and solved
the problems if they encountered the case described in the video (Choi et al., 2013, 2009; Choi &
Lee, 2009). The different specialties of the faculty members— three of the faculty members
were Diplomates in the American College of Veterinary Surgery, and one member was
Diplomate in the American College of Veterinary Internal Medicine—allowed diverse
approaches to the given problem. The four experts did not always approach the given problem in
a similar way. The four experts sometimes identified different solutions. Given that an ill-
defined clinical setting is characterized by the involvement of multiple players (Higgs & Jones,
2008; Orasanu & Connolly, 1993), these diverse specialties were helpful to better train students.
45
At each Decision Point, two or three experts’ commentary videos were embedded. Each
video lasted three to five minutes. Transcripts of the interview videos were delivered in a PDF
form along with the videos.
Scaffolded Revision of the Initial Clinical Decision
In order to enhance veterinary students’ clinical decision-making skills by promoting
their reflection, the participants in this study were encouraged to revisit and make revisions on
their initial decisions while learning with the case-based online learning module (Choi et al.,
2013, 2009; Choi & Lee, 2009).
To better help their revision, they received opportunities to compare their opinions with
others’ opinions before making revisions. Thus, the specific pattern of the activity was [Stage 1:
Making an initial decision] – [Stage 2: Watching expert commentary] – [Stage 3: Exchanging
peer feedback] – [Stage 4: Revising the initial decision-making]. Stages 1, 2 and 4 were
mandatory for all participants, and Stage 3 was optional for those who participated in either early
or later peer feedback session. This pattern was repeated by each Decision Point.
Stage 1: Making an Initial Decision
In the first step of the learning activity, the students watched a short clinical case video.
Based on the case video, the students were asked to identify problems and make an action plan
for the patient. Four critical thinking prompts were provided to assist their decision-making
process of identifying key information, assessing the current case, prioritizing issues and
objectives, and making an immediate plan. The patient’s Medical Records was provided in case
they need more information about the patient. The Digital Textbook including relevant medical
knowledge from textbook and journal articles was also provided to support the students’ content
46
knowledge. After submitting their answers, they were led to the next step, watching the expert
commentary videos.
Stage 2: Watching Expert Commentary
In the next step, the two or three experts’ commentary videos were presented. In the
videos, the experts narrated about their approaches to the situation presented in the case video.
The videos showed the key characteristics of the expert decision-making process: how they
recognize meaningful cues, assess situation(s), set goals, generate solution(s) and justify the
solution(s). By watching the expert videos, the students had a chance to compare their initial
decision-making responses and those of experts.
Stage 3: Exchanging Peer Feedback
After watching expert commentary videos and before making revisions on their initial
clinical decisions, students were allowed to participate in a peer feedback session. A total of two
peer feedback sessions were available, and participants had a chance to self-select to join either
one or no. In each peer feedback session, the participants completed two Decision Points
(Decision Points 2 and 3 in the first session and Decision 4 and 5 for the second session) with
their partner.
In each peer feedback session, students were encouraged to individually complete the
previous steps, making initial decisions and watching expert commentary videos. Then, they
were asked to share their initial clinical decisions with their partner and discuss how their initial
decisions were similar and different from those of experts and how to revise their responses. To
facilitate the discussion with partner, a worksheet with guidelines for the session and reflective
prompts were provided (See Appendix A for more information). The reflective prompts guided
the students in comparing their partner’s initial responses to expert commentary (e.g., What are
47
the similarities and differences between the experts’ identification of key information and that of
your partner? Are any important cues missing from your partner’s list?). After sharing feedback
with their partner, the participants were allowed to move onto the next step, revising initial
decisions.
Stage 4: Revising the Initial Decision
After reviewing the experts’ commentary videos, the students were allowed to revise
their initial answers. This revision activity was designed to encourage students to reflect on their
initial answers. To help their revision, their initial answers were automatically retrieved. The
patient medical records and relevant information were still available. After submitting the
revised answers, they were led to the next Decision Point, which had the same pattern of the
learning activity.
Procedures
Data collection was conducted for approximately nine weeks, from early October to mid
December. In the mid October, the instructor of the course had a demonstration session to
introduce the learning module including the purpose of the module, how to access and navigate
the module, and how to learn with the module. Students were asked to complete this learning
module by the second week of December. The instructor also provided information about two
optional peer feedback sessions: students were allowed to self-select which session they would
like to attend. Over twenty students voluntarily registered for each peer feedback session.
The first scaffolded peer feedback session was conducted on the last week of October, in
two weeks from the demonstration session. Decision Point 2 and 3 were assigned to the first
peer feedback session, and an hour was assigned for each Decision Point. Before session began,
one of the veterinary faculty members reminded the participants of the information about the
48
session including the purpose and procedures of the session. The participants then received a
printed worksheet with the general information of the session including the purpose and
procedure. The worksheet also included question prompts to support the participants’ peer
feedback activity (e.g., What are the similarities and differences between the experts’
identification of key information and that of your partner? Are any important cues missing from
your partner’s list?). The participants were asked to individually make an initial clinical decision
and watch expert commentary videos. Then, they had a 10- or 15-minute of exchanging
feedback with their partner based on the question prompts. They reviewed their partner’s initial
clinical decision, compared the expert clinical decision, and suggested how to revise the initial
decision. Then they were asked to individually revise their initial responses based on the expert
commentary videos and feedback they received from their partner.
The second scaffolded peer feedback session was conducted in mid November, in two
weeks apart from the first peer feedback session. The second session was also conducted at the
same place in which the first peer feedback session occurred. The instructor of the course
reminded the participants of the general information about the session including the purpose and
procedures of the session before the session began. The same worksheets used for the first
session were also provided. The participants individually as well as collaboratively completed
the assigned two Decision Points during the two-hour session, as same as the participants did in
the first peer feedback session.
Upon the completion of the learning module, an online survey embedded in the very last
closing part was distributed to all participants in order to explore their learning experiences.
Also, three face-to-face interviews with three volunteers were conducted: one interviewee
49
participated in the first peer feedback session (EC/EP group) and the other two interviewees
participated in the second peer feedback session (EC/LP group).
Research Design
The purpose of this study was to attempt to enhance veterinary students’ clinical decision-
making skills through scaffolded revision activities in a case-based learning environment.
Specifically, the scaffolded revision activities in a case-based learning environment were
designed to promote the students’ knowledge acquisition and reflection on their thinking and
actions.
Research Questions
This research was designed to answer the following research questions:
Research Question 1. Do the scaffolded revision activities enhance the quality of
students’ revised clinical decisions in case-based learning?
o RQ1-1. Do the scaffolded revision activities enhance the overall quality of
students’ revised clinical decision in case-based learning?
o RQ1-2. Do the scaffolded revision activities enhance the quality of students’
revised case assessment in case-based learning?
o RQ1-3. Do the scaffolded revision activities enhance the overall quality of
students’ revised prioritization of issues and objectives in case-based learning?
o RQ1-4. Do the scaffolded revision activities enhance the overall quality of
students’ revised plan of an immediate action in case-based learning?
Research Question 2. Do the scaffolded revision activities enhance the quality of
students’ revised clinical decision in case-based learning?
50
o RQ 2-1. Are there significant differences in the quality of students’ initial and
revised clinical decision among the expert commentary only group (EC/NP),
expert commentary with early peer feedback group (EC/EP), and expert
commentary with later peer feedback group (EC/LP)?
o RQ 2-2. Are there significant differences in the quality of students’ initial and
revised clinical decision between peer feedback groups (EC/EP and EC/LP) and
no peer feedback group (EC/NP)?
o RQ2-3. Are there significant differences in the quality of students’ initial and
revised clinical decision between early peer feedback group (EC/EP) and later
peer feedback group (EC/LP)?
Research Question 3. Are there significant differences in the quality of the initial and
revised clinical decision among groups across the two peer feedback sessions?
o RQ 3-1. Are there significant differences in the quality of students’ initial and
revised clinical decision among the expert commentary only group (EC/NP),
expert commentary with early peer feedback group (EC/EP), and expert
commentary with later peer feedback group (EC/LP) across the two peer feedback
sessions?
o RQ 3-2. Are there significant differences in the quality of students’ initial and
revised clinical decision between peer feedback groups (EC/EP and EC/LP) and
no peer feedback group (EC/NP) across the two peer feedback sessions?
o RQ 3-3. Are there significant differences in the quality of students’ initial and
revised clinical decision between early peer feedback group (EC/EP) and later
peer feedback group (EC/LP) across the two peer feedback sessions?
51
Research Question 4. Does the participation in a scaffolded revision activity affect
students’ transferred clinical decision-making skills?
o RQ 4-1. Are there significant differences in the scores on a transferred clinical
decision test among EC/NP (expert commentary only), EC/EP (expert
commentary and early peer feedback), and EC/LP (expert commentary and later
peer feedback)?
o RQ 4-2. Are there significant differences in the scores on a transferred clinical
decision test between peer feedback groups (EC/EP and EC/LP) and no peer
feedback group (EC/NP)?
o RQ 4-3. Are there significant differences in the scores on a transferred clinical
decision test between early peer feedback group (EC/EP) and later peer feedback
group (EC/LP)?
Research Question 5. What are the students’ perceptions on the revision activity in the
case-based online learning module?
o RQ 5-1. What are the students’ perceptions on the expert commentary for revising
their initial clinical decisions?
o RQ 5-2. What are the students’ perceptions on the peer feedback for revising their
initial clinical decisions?
o RQ 5-3. What are the students’ perceptions on the effectiveness of the peer
feedback compared to the expert commentary?
A Time-Series Design
A time-series design was used to evaluate the effectiveness of the scaffolded peer
feedback and its timing. A time-series design typically involves multiple observations of data
52
gathered both prior to and after the intervention (Shadish, Cook, & Campbell, 2002). This
design is based on the idea that researchers are able to identify a pattern by observing data both
prior to and after an intervention. A pattern may show an ongoing increase, decline, flat or
fluctuating. By identifying the pattern before and after the intervention, a researcher is able to
examine the effect of the intervention. The design for this study is diagrammed as in Table 3-3.
Table 3-3
A time series design for this research
S1 (October, 2015) S2 (November, 2015) S3 (December, 2015)
EC/NP O1-Pre X1 O1-Post O2-Pre X1 O2-Post O3
EC/EP O1-Pre X2 O1-Post O2-Pre X1 O2-Post O3
EC/LP O1-Pre X1 O1-Post O2-Pre X2 O2-Post O3
Note. EC/NP, EC/EP, and EC/LP indicate the three groups of the participants in this study:
EC/NP received expert commentary only, EC/EP received expert commentary and early peer
feedback, while EC/LP received expert commentary and later peer feedback. S1, S2 and S3
indicate the three sessions of data collection to measure students’ clinical decision-making skills.
S1 and S2 consist of two decision points embedded in the case-based e-learning environment
respectively. During S1 and S2, a group with peer feedback spent two hours to complete the
assigned two modules, whereas the other groups who did not receive peer feedback
independently spent two weeks to complete the modules. For example, During S1, the EC/EP
group spent two hours to complete the module, while the other groups of EC/NP and EC/LP
independently completed the module within two weeks. S3 took approximately 20 minutes to
complete. O1 to O3 indicate data collections to measure students’ clinical decision-making skills.
O1-Pre and O2-pre indicate students’ initial clinical decisions made in the corresponding session.
O1-Post and O2-Post indicate students’ revised clinical decisions made in the corresponding session.
O3 indicates multiple-choice exam designed to measure students’ near-transferred clinical
decision-making skills. X1 indicates the treatment, scaffolded revision with expert commentary
only, and X2 indicates the treatment, scaffolded revision with expert commentary as well as peer
feedback.
Data Collection and Analysis
To answer the research questions, multiple data were collected and analyzed accordingly.
Data collection and analysis plan is summarized in Table 3-4.
53
Table 3-4
The research questions, data collection, data source, and data analysis techniques
Research Question Data Collection Data Source Analysis
Technique
RQ1. Does the expert
commentary enhance the
quality of students’ revised
clinical decisions in case-
based learning?
A pre-post design
S1 and S2
ALL Opre X Opost
Assessment,
prioritization, plan,
and average scores
of the student
initial and revised
decision-making
responses
Repeated
ANOVAs
RQ2-1. Are there significant
differences in the quality of
students’ initial and revised
clinical decision among the
expert commentary only
group (EC/NP), expert
commentary with early peer
feedback group (EC/EP), and
expert commentary with later
peer feedback group
(EC/LP)?
A time series with
switching replications
design
S1 and S2
EC/NP Opre X1 Opost
EC/EP Opre X2 Opost
EC/LP Opre X2 Opost
Assessment,
prioritization, plan,
and average scores
of the student
initial and revised
decision-making
responses
Repeated-
measures
ANOVAs
RQ2-2. Are there significant
differences in the quality of
students’ initial and revised
clinical decision between
peer feedback groups (EC/EP
and EC/LP) and no peer
feedback group (EC/NP)?
A time series with
switching replications
design
S1 and S2
EC/NP Opre X1 Opost
EC/EP
&
EC/LP
Opre X2 Opost
Assessment,
prioritization, plan,
and average scores
of the student
initial and revised
decision-making
responses
Repeated-
measures
ANOVAs
RQ2-3. Are there significant
differences in the quality of
students’ initial and revised
clinical decision between
early peer feedback group
(EC/EP) and later peer
feedback group (EC/LP)?
A time series with
switching replications
design
S1 and S2
EC/EP Opre X Opost
EC/LP Opre X Opost
Assessment,
prioritization, plan,
and average scores
of the student
initial and revised
decision-making
responses
Repeated-
measures
ANOVAs
54
Research Question Data Collection Data Source Analysis
Technique
RQ3-1. Are there significant
differences in the quality of
students’ initial and revised
clinical decision among the
expert commentary only
group (EC/NP), expert
commentary with early peer
feedback group (EC/EP), and
expert commentary with later
peer feedback group (EC/LP)
across the two peer feedback
sessions?
A time series with
switching replications
design
S1 S2
EC/NP O X1 O O X1
O
EC/EP O X2 O O X1
O
EC/LP O X1 O O X2
O
Assessment,
prioritization, plan,
and average scores
of the student
initial and revised
decision-making
responses
Repeated-
measures
ANOVAs
RQ3-2. Are there significant
differences in the quality of
students’ initial and revised
clinical decision between
peer feedback groups (EC/EP
and EC/LP) and no peer
feedback group (EC/NP)
across the two peer feedback
sessions?
A time series with
switching replications
design
S1 S2
EC/NP O X1 O O X1
O
EC/EP
&
EC/LP
O X2 O O X2
O
Assessment,
prioritization, plan,
and average scores
of the student
initial and revised
decision-making
responses
Repeated-
measures
ANOVAs
RQ3-3. Are there significant
differences in the quality of
students’ initial and revised
clinical decision between
early peer feedback group
(EC/EP) and later peer
feedback group (EC/LP)
across the two peer feedback
sessions?
A time series with
switching replications
design
S1 S2
EC/EP O X2 O O X1
O
EC/LP O X1 O O X2
O
Assessment,
prioritization, plan,
and average scores
of the student
initial and revised
decision-making
responses
Repeated-
measures
ANOVAs
55
Research Question Data Collection Data Source Analysis
Technique
RQ4-1. Are there significant
differences in the scores on a
transferred clinical decision
test among EC/NP (expert
commentary only), EC/EP
(expert commentary and
early peer feedback), and
EC/LP (expert commentary
and later peer feedback)?
A posttest-only design
S1 S2 S3
EC/NP X1 X2 O
EC/EP X2 X1 O
EC/LP X1 X2 O
A multiple-choice
transfer test
One-way
ANOVA
RQ4-2. Are there significant
differences in the scores on a
transferred clinical decision
test between peer feedback
groups (EC/EP and EC/LP)
and no peer feedback group
(EC/NP)?
A posttest-only design
S1 S2 S3
EC/NP X1 X2 O
EC/EP
&
EC/LP
X2 X2 O
A multiple-choice
transfer test
One-way
ANOVA
RQ4-3. Are there significant
differences in the scores on a
transferred clinical decision
test between early peer
feedback group (EC/EP) and
later peer feedback group
(EC/LP)?
A posttest-only design
S1 S2 S3
EC/EP X2 X1 O
EC/LP X1 X2 O
A multiple-choice
transfer test
One-way
ANOVA
RQ5. What are the students’
perceptions on the revision
activity in the case-based
online learning module?
- Online survey and
face-to-face
interviews
-
Clinical decision-making skills
Student initial and revised answers were stored in the web-based data server and
collected to examine students’ clinical decision-making skills. In order to analyze the written
responses that the students generated at each Decision Point, three dimensions were identified as
dependent variables (see Table 3-5).
56
Table 3-5
Three dimensions of the clinical decision-making skills and their rubric
Clinical decision-
making skills
Rubric
Assessment Accurately interprets value of chosen evidence, findings, data, etc.;
Identifies the strengths and weaknesses of each piece of key information;
Thoughtfully analyzes and evaluates content of key information;
Draws warranted, judicious, appropriate conclusions.
Objectives Identifies the strengths and weaknesses of each piece of key information;
Thoughtfully analyzes and evaluates obvious alternative points of view;
Commits to a judicious priority system and explains assumptions and
reasons.
Choice
Explanation
Thoughtfully analyzes and evaluates content of key information;
Commits to chosen course of action and explains assumptions and
reasons; Addresses the potential for error by proposing a plan for
management of error;
Fair-mindedly follows where evidence and reasons lead.
In order to analyze the quality of students’ clinical decision-making skills, the same panel
of veterinary educators identified ideal decision-making scripts of each Decision Point. Based
on the ideal scripts, the educators graded each student’s initial and revised responses.
Figure 3-4
Scoring system for student decision-making responses
57
The rules that the veterinary educators used to grade students’ answers are as follows (see
Figure 3-4). First, the educators checked whether a student chose appropriate answers to a
corresponding multiple-choice question. 25% or fewer score was graded when the student chose
wrong answer, and 25% or more was given when the student chose appropriate answers. In case
of appropriate answers, the student’s score depended on his/her justification. When the
justification was poor, the maximum score given was 50%. To elaborate, if the student did not
provide any justification, 50% was given. If the student showed wrong justification, 25% score
would be maximum. In case of noncommittal, or accurate description but not related to the
question, 30% score was given. In case of appropriate multiple-choice answers as well as
justifications, 50% or more score was given. The student’s final score was determined based on
key variables that the student considered in justifying his/her decision-making. Table 3-6
presents an example of key variables that a student is supposed to consider and the ideal answers
of Decision Point 2.
58
Table 3-6
Sample ideal script at Decision Point 2 “Taking Action for Doug”
Decision-making
Process
What must consider Ideal scripts
Assessing the
current case
1. Description of the
intestine
2. Antimesenteric
enterotomy
3. Indications for a
resection and
anastomosis
4. Perforation
5. No resection is
required
The intestine appears healthy at the site of
the obstruction. While you can see
distention in the area of the foreign body, the
intestinal color is normal and there is no
perforation. Because the intestine appears
healthy, an antimesenteric enterotomy is
indicated to remove the foreign material and
relieve the obstruction. A resection and
anastomosis is not indicated. Indications for
a resection and anastomosis include
devitalized bowel evidence by a dark color
(black or purple) or perforation, neither is
observed. Additionally there is no free fluid
or evidence of peritonitis or serositis.
Prioritizing
issues and
objectives
1. No prognosis
2. Re-establishment of
normal aboral flow of
ingesta
The major objective is to relieve the
intestinal obstruction and re-establish normal
aboral flow of ingesta. The prognosis for
Doug is good because the intestine appears
healthy and no resection is required.
Planning an
immediate action
1. Enteronomy
2. Viability of the bowel
3. Foreign body cannot
be removed
Simple enterotomy is chosen because the
foreign body cannot be moved, but the
bowel is not sufficiently devitalized to
warrant either resection/anastomosis or
euthanasia.
A total of three veterinary educators reviewed similarities between student responses and
the ideal scripts in terms of their correctness and justifications. Because Decision Point 1 was
used for the demonstration session, students’ responses at Decision Point 1 were excluded from
the review. For each Decision Point, two trained raters (veterinary educators) reviewed students’
answers: reviewer A and B were assigned at Decision Point 2, Reviewer A and C were assigned
59
at Decision Point 3, Reviewer C and B were assigned at Decision Point 4, and Reviewer B and C
were assigned at Decision Point 5. The experts independently reviewed the blind data which
were coded with random numbers for students’ names. Each student received in total of four
scores: scores for assessing case, prioritizing issues and objectives, and planning as well as
overall average of the three scores. The scoring range for student decision-making responses
was between 0 and 100.
In order to calculate the inter-rater reliabilities, 10% of student answers were randomly
selected from each Decision Point. Two trained raters (veterinary educators) evaluated the blind
data independently. When there were big discrepancies (e.g., more than threshold scores:
average score + SD score) in students’ scores given by each of the two raters, scores for these
answers were negotiated and newly identified. For each variable, Pearson’s r was calculated
over the scores of the reviewers as an inter-rater reliability coefficient. The initial average
reliability was r = .769 (p < .01) and the negotiated average reliability was r = .927 (p < .01) as
indicated in Table 3-7. Since the average inter-rater reliability was reached at an acceptable level,
the first rater proceeded to evaluation of the remaining answers. For the analysis, the scores of
the first rater were used.
Table 3-7
Inter-rater reliabilities over the Decision Points
Decision
Point 2
Decision
Point 3
Decision
Point 4
Decision
Point 5
Overall
Before
negotiation
.833**
.820**
.795**
.641**
.769**
After
negotiation
.943**
.937**
.957**
.811**
.927**
* p < .05, ** p < .01
60
Transferred clinical decision-making skills
To examine students’ transferred clinical decision-making skills, a final test was
conducted upon the completion of the learning module. The final test was embedded at the
conclusion of the learning module. The test included questions about gastrointestinal disease
cases with different symptoms from the case embedded in the case-based learning environment.
The test began with a written case including clinical history and results of physical
examination. The test consisted of six multiple-choice questions which asked students to make a
series of clinical decisions. Scores are given if a choice is correct. Figure 3-5 shows one of the
six questions.
Figure 3-5
A sample question from the final test
Learning experience survey
To determine students’ appreciation of the peer feedback session, the reflection prompts,
and the case-based learning module, the student opinions were collected by means of an
evaluation questionnaire. The questionnaire was embedded in the module, at the closing
Decision Point.
The questionnaire comprises of three groups: learning experiences with expert
commentary only, learning experiences with expert commentary and peer feedback, and
perception on the effectiveness between learning experiences with expert commentary only and
61
those with an addition of peer feedback. The participants who participated in any peer feedback
session received all three groups of questions, and the participants who did not participate in any
feedback session received the first group of questions only.
For each group of questions, the students could indicate to what extent they agree to it.
Most items were accompanied by a 5-point Likert scale, raining from 1 being strongly disagree
to 5 being strongly agree. Additionally, several follow-up open-ended questions were posed to
deeply investigate students’ experiences with the scaffolded revision activities. The online
survey is appended as Appendix A.
Learning experience interview
To explore students’ experiences with the peer feedback sessions in detail, individual
interviews were conducted after the learning module was completed. Those who participated in
the peer feedback sessions were recruited for the interviews via email with the instructor’s help.
A total of three students voluntarily agreed to participate in the interviews. Interviewee A was a
female student, who participated in the first peer feedback session. Interviewee B and C were
female students, who participated in the second peer feedback session.
During the one-hour face-to-face interviews, the participants were asked to describe their
clinical decision-making processes when doing alone with expert commentary only and doing
with the addition of their partner’s feedback. Specifically, they were asked to describe how they
decided to participate in the peer feedback session, what activities they expected at first, and how
the expectations were different from the real feedback session.
Moreover, they were asked to describe how they made an initial clinical decision and
revised it during learning the module. Also, they were asked to elaborate on what they did when
participating in the peer feedback session, such as how and what they discussed with their
62
partner and revised their initial responses. The interviews were audio-recorded with the
interviewees’ permission and transcribed for further analysis.
63
CHAPTER 4
RESULTS
For this research, the three groups’ decision-making scores were collected as presented in
Table 4-1. To examine the effectiveness of the expert commentary videos, peer feedback, timing
of the peer feedback, and interaction effects of the three variables, different combinations of the
data were used.
Table 4-1
Data collection for the quality of student decision-making across groups in three sessions
Session 1 Session 2 Across Sessions 1 & 2 Session 3
Initial Revised Initial Revised Initial Revised Transfer
test
EC/NPa NP1-1 NP1-2 NP2-1 NP2-2 NP-1 NP-2 NP-3
EC/EPb EP1-1 EP1-2 EP2-1 EP2-2 EP-1 EP-2 EP-3
EC/LPc LP1-1 LP1-2 LP2-1 LP2-2 LP-1 LP-2 LP-3
Total SS1-1 SS1-2 SS2-1 SS2-2 AVG-1 AVG-2
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
RQ1. Revision effects of the scaffolded revision activities
Research Question 1. Do the scaffolded revision activities enhance the quality of
students’ revised clinical decision in case-based learning?
o Research Question 1-1. Do the scaffolded revision activities enhance the overall
quality of students’ revised clinical decision in case-based learning?
64
o Research Question 1-2. Do the scaffolded revision activities enhance the quality
of students’ revised case assessment in case-based learning?
o Research Question 1-3. Do the scaffolded revision activities enhance the overall
quality of students’ revised prioritization of issues and objectives in case-based
learning?
o Research Question 1-4. Do the scaffolded revision activities enhance the overall
quality of students’ revised plan of an immediate action in case-based learning?
The first research question examines the effects of the scaffolded revision activities on
the quality of students’ revised clinical decision in a case-based learning environment. In order
to test whether the quality of the revised clinical decision was significantly improved after
participating in the scaffolded revision activities, all participants’ overall initial and revised
scores from the two data collection sessions were collected. Specifically, the overall quality of
all participants’ initial and revised decisions was tested. If there were significant differences
between the initial and revised decisions, a follow-up test on the three sub-dimensions (i.e., case
assessment, prioritization of issues and objectives, and plan of an immediate action) would be
tested.
65
Table 4-2
Data used to test the revision effect (Research Question 1)
Session 1 Session 2 Across Sessions 1 & 2 Session 3
Initial Revised Initial Revised Initial Revised Transfer
test
EC/NPa - - - - - - -
EC/EPb - - - - - - -
EC/LPc - - - - - - -
Total - - - - AVG-1 AVG-2 -
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
RQ1-1. Revision effects on the overall quality of the revised clinical decision (Initial vs.
Revised clinical decision)
Hypothesis 1-1. The overall quality of students’ revised clinical decision after
participating in the scaffolded revision activities will be significantly improved than the overall
quality of the initial clinical decision before watching the expert commentary videos in case-
based learning.
Descriptive statistics on the initial and revised decisions from the two sessions are
presented in Table 4-3. As shown in Table 4-3, the average score of initial decisions was 47.61
(SD = 11.23), and the average score of revised decisions was 54.52 (SD = 11.54). Based on the
descriptive data in Table 4-3, a graphical comparison in the overall quality of the initial and
revised clinical decisions is presented in Figure 4-1.
66
Table 4-3
Descriptive statistics on the quality of the initial and revised case assessment
(N = 47)
Initial Revised
M SD M SD
Overall 47.61 11.23 54.52 11.54
Figure 4-1
The overall quality of initial and revised clinical decisions
To examine whether the quality of students’ initial answers were significantly improved
after participating in the scaffolded revision activities, a repeated-measures ANOVA with the
overall quality of the students’ initial and revised clinical decisions was conducted (Table 4-4).
67
Table 4-4
Summary of repeated-measures ANOVA for the overall quality of the initial and revised clinical
decisions
Source Sum of Squares df Mean square F p η2
Within subjects
Revisiona 2241.582 1 2241.582 44.081 .000 .489
Error (Revision) 2339.175 46 50.852
Between subjects
Intercept 122548.936 1 122548.936 1047.263 .000 .958
Error 5382.843 46 117.018
Note. aRevision indicates the quality improvement of the revised clinical decision after
watching the expert commentary videos.
As described in Table 4-4, the results revealed a significant main effect, F(1, 46) =
44.081, p < .05, indicating that there were statistically significant differences in the quality
between the initial (M = 47.61, SD = 11.23) and revised decisions (M = 54.52, SD = 11.54). In
other words, the overall quality of the initial decision-making was significantly improved after
participating in the scaffolded revision activities. The difference between the initial and revised
answers was a large effect, with η2 = .489 (Cohen, 1988).
RQ1-2. Revision effects on the quality of assessment (Initial vs. Revised)
Hypothesis 1-2. The quality of students’ revised case assessment after participating in the
scaffolded revision activities will be significantly improved than the quality of the initial case
assessment before participating in the scaffolded revision activities.
Since the overall quality of the revised clinical decisions was significantly enhanced after
participating in the scaffolded revision activities, follow-up tests were conducted to examine
whether significant differences among the three sub-dimensions exist. In the follow-up analysis,
68
multiple repeated-measures ANOVAs were employed to compare the quality of the initial and
revised clinical decisions in terms of the three dimensions—case assessment, prioritization of
issues and objectives, and plan of an immediate action.
Descriptive statistics on the initial and revised case assessment are described in Table 4-
5. As shown in Table 4-5, the average score of initial case assessment was 46.00 (SD = 14.37),
and the average score of revised decisions was 50.60 (SD = 15.70).
Table 4-5
Descriptive statistics on the quality of the initial and revised case assessment
(N = 47)
Initial Revised
M SD M SD
Assessment 46.00 14.37 50.60 15.70
In order to identify whether the differences between the initial and revised case
assessment are statistically significant, the qualities of all participants’ responses in case
assessment were analyzed using a repeated measures ANOVA with revision (initial answers and
revised answers) as a within-subjects factor.
69
Table 4-6
Summary of repeated-measures ANOVA for the qualities of the initial and revised case
assessment
Source Sum of Squares df Mean
square
F p η2
Within subjects
Revisiona 994.980 1 994.980 10.714 .002 .189
Error (Revision) 4272.082 46 92.871
Between subjects
Intercept 109624.096 1 109624.096 539.469 .000 .921
Error 9347.545 46 203.207
Note. aRevision indicates the qualities of the initial and revised clinical decision.
As described in Table 4-6, the main effect of revision was significant, F(1, 46) = 10.714,
p < .05. This result indicated that there were statistically significant differences between the
qualities of the initial (M = 46.00, SD = 14.37) and revised case assessment (M = 50.60, SD =
15.70). In other words, the overall quality of the students’ revised case assessment was
significantly improved than the initial case the quality of the initial case assessment. The
difference between the initial and revised case assessment was a small effect, with η2 = .189
(Cohen, 1988).
RQ1-3. Revision effects on the quality of prioritization (Initial vs. Revised)
Hypothesis 1-3. The quality of students’ revised prioritization of issues and objectives
after participating in the scaffolded revision activities will be significantly improved than the
quality of the initial prioritization before participating in the scaffolded revision activities.
70
Descriptive statistics on the initial and revised prioritization of issues and objectives are
described in Table 4-7. As shown in Table 4-7, the average score of initial prioritization was
46.68 (SD = 11.49), and the average score of revised decisions was 52.29 (SD = 13.05).
Table 4-7
Descriptive statistics on the quality of the initial and revised prioritization of issues and
objectives
(N = 47)
Initial Revised
M SD M SD
Prioritization 46.68 11.49 52.29 13.05
To identify whether the differences between the initial and revised prioritization of issues
and objectives are statistically significant, the qualities of all participants’ responses in
prioritization were analyzed using a repeated measures ANOVA with revision (initial answers
and revised answers) as a within-subjects factor.
71
Table 4-8
Summary of repeated-measures ANOVA for the qualities of the initial and revised prioritization
of issues and objectives
Source Sum of Squares df Mean square F p η2
Within subjects
Revisiona 1480.086 1 1480.086 25.294 .000 .355
Error (Revision) 2691.726 46 58.516
Between subjects
Intercept 115075.141 1 115075.141 842.958 .000 .948
Error 6279.624 46 136.514
Note. aRevision indicates the qualities of the initial and revised clinical decision.
As described in Table 4-8, the main effect of revision was significant, F(1, 46) = 25.294,
p < .05. This result indicated that there were statistically significant differences between the
qualities of the initial (M = 46.68, SD = 11.49) and revised prioritization (M = 52.59, SD =
13.05). In other words, the overall quality of the students’ revised prioritization of issues and
objectives was significantly improved than the initial case the quality of the initial prioritization.
The difference between the initial and revised prioritization was a medium effect, with η2 = .355
(Cohen, 1988).
RQ1-4. Revision effects on the quality of plan (Initial vs. Revised)
Hypothesis 1-4. The quality of students’ revised plan of an immediate action after
participating in the scaffolded revision activities will be significantly improved than the quality
of the initial plan before participating in the scaffolded revision activities.
72
Descriptive statistics on the quality of the initial and revised plan of an immediate action
are summarized in Table 4-9. As shown in Table 4-9, the average score of initial plan was 50.16
(SD = 13.62), and the average score of revised decisions was 60.66 (SD = 13.30).
Table 4-9
Descriptive statistics on the quality of the initial and revised plan of an immediate action
(N = 47)
Initial Revised
M SD M SD
Plan 50.16 13.62 60.66 13.30
To identify whether the differences between the initial and revised plan of an immediate
action are statistically significant, the qualities of all participants’ responses in plan were
analyzed using a repeated measures ANOVA with revision (initial answers and revised answers)
as a within-subjects factor.
Table 4-10
Summary of repeated-measures ANOVA for the quality of the initial and revised plan of an
immediate action
Source Sum of Squares df Mean
square
F p η2
Within subjects
Revisiona 5187.001 1 5187.001 40.124 .000 .466
Error (Revision) 5946.561 46 129.273
Between subjects
Intercept 144314.237 1 144314.237 969.175 .000 .955
Error 6849.591 46 148.904
Note. aRevision indicates the qualities of the initial and revised clinical decision.
73
As described in Table 4-10, the main effect of revision was significant, F(1, 46) = 40.124,
p < .05. This result indicated that there were statistically significant differences between the
qualities of the initial (M = 50.16, SD = 13.62) and revised prioritization (M = 60.66, SD =
13.30). In other words, the overall quality of the students’ revised plan of an immediate action
was significantly improved than the initial case the quality of the initial plan. The difference
between the initial and revised plan was a large effect, with η2 = .466 (Cohen, 1988). Based on
the descriptive data, a graphical comparison in the quality of the three sub-dimensions of initial
and revised clinical decisions is presented in Figure 4-2.
Figure 4-2
The quality of the three sub-dimensions of the initial and revised clinical decisions
RQ2. Revision effects by groups
Research Question 2. Do the scaffolded revision activities enhance the quality of
students’ revised clinical decision in case-based learning?
74
o Research Question 2-1. Are there significant differences in the quality of students’
initial and revised clinical decision among the expert commentary only group
(EC/NP), expert commentary with early peer feedback group (EC/EP), and expert
commentary with later peer feedback group (EC/LP)?
o Research Question 2-2. Are there significant differences in the quality of students’
initial and revised clinical decision between peer feedback groups (EC/EP and
EC/LP) and no peer feedback group (EC/NP)?
o Research Question 2-3. Are there significant differences in the quality of students’
initial and revised clinical decision between early peer feedback group (EC/EP)
and later peer feedback group (EC/LP)?
The second research question examines the quality of the initial and revised clinical
decisions among the groups based on the three diverse scaffolded revision activities. To test the
interaction effect between revision and group, multiple repeated-measures ANOVAs were
conducted with three different group variations: (1) the first comparison was among EC/NP,
EC/EP, EC/LP groups on the basis of the quality of the initial and revised clinical decisions; (2)
the second comparison was between the EC/NP group with expert commentary only to the other
two groups, EC/EP and EC/LP, with the addition of peer feedback; and (3) the last comparison
was between the two peer feedback groups, EC/EP and EC/LP, depending on the timing of the
peer feedback.
As presented in Table 4-11, students’ overall initial and revised scores across three
groups were collected to address the Research Question 2. For each of the sub-questions under
Research Question 2, the overall quality of all participants’ initial and revised decisions was
tested. If there were significant differences between the initial and revised decisions, a follow-up
75
test on the three sub-dimensions (i.e., case assessment, prioritization of issues and objectives, and
plan of an immediate action) would be tested.
Table 4-11
Data used to test the two-way interaction effect of revision and group (Research Question 2)
Session 1 Session 2 Across Sessions 1 & 2 Session 3
Initial Revised Initial Revised Initial Revised Transfer
test
EC/NPa - - - - NP-1 NP-2 -
EC/EPb - - - - EP-1 EP-2 -
EC/LPc - - - - LP-1 LP-2 -
Total - - - - - -
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
RQ2-1. Revision effects among groups (EC/NP vs. EC/EP vs. EC/LP)
Hypothesis 2-1. There will be significant differences between the qualities of initial and
revised clinical decision among the expert commentary only group (EC/NP), expert commentary
with early peer feedback group (EC/EP), and expert commentary with later peer feedback group
(EC/LP) in case-based learning.
To test the Hypothesis 2-1, the initial and revised clinical decision scores of the
participants in the three groups were collected. A total of four different kinds of scores on the
quality of the clinical decision were calculated: scores on the quality of the three sub-dimensions
(case assessment, prioritization of issues and objectives, and plan of an immediate action) and
overall average of the three sub-dimensions scores.
RQ2-1-1. On the overall quality of the decision (EC/NP vs. EC/EP vs. EC/LP).
Hypothesis 2-1-1. There will be significant differences between the overall quality of the initial
76
and revised clinical decision among the expert commentary only group (EC/NP), expert
commentary with early peer feedback group (EC/EP), and expert commentary with later peer
feedback group (EC/LP) in case-based learning.
Descriptive statistics on the quality of the initial and revised decisions of the participants
in the EC/NP, EC/EP, and EC/LP are summarized in Table 4-12. As shown in Table 4-12, the
average scores of initial decision of the EC/NP, EC/EP, and EC/LP are 45.30 (SD = 11.02),
51.31 (SD = 12.47), and 49.49 (SD = 10.59), respectively. The average scores of revised
decision of EC/NP, EC/EP, and EC/LP are 50.82 (SD = 12.93), 59.76 (SD = 7.42), and 57.99
(SD = 8.82), respectively.
Table 4-12
Descriptive statistics on the overall quality of the initial and revised clinical decisions among
EC/NP, EC/EP, and EC/LP
Initial Revised
M SD M SD
EC/NPa (n=25) 45.30 11.02 50.82 12.93
EC/EPb (n=9) 51.31 12.47 59.76 7.42
EC/LPc (n=13) 49.49 10.59 57.99 8.82
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
To examine whether the differences between the initial and revised decisions by three
groups were significant, a repeated-measures ANOVA was conducted (see Table 4-13). The
results revealed a significant revision effect (F = 43.541, p < .05) indicating that there were
statistically significant differences between the initial and revised answers. In other words, the
overall quality of the initial answers was significantly improved through the revision in the case-
77
based online module. The difference between the initial and revised answers was a large effect,
with η2 = .497 (Cohen, 1988).
On the other hand, the interaction effect between revision and group was not significant
(F = 1.004, p > .05) indicating that the differences in the quality of the initial and revised clinical
decisions were similar across the groups. In other words, the overall quality of the revised
decisions made by the three groups was significantly improved, but no significant differences
among groups existed. Thus, hypothesis 2-1 which assumes different quality of the initial and
revised clinical decisions across the three groups was denied.
Table 4-13
Summary of repeated-measures ANOVA for the overall quality of the initial and revised clinical
decisions in EC/NP, EC/EP, and EC/LP
Source Sum of
Squares
df Mean
square
F p η2
Within subjects
Revisiona 2213.708 1 2213.708 43.541 .000 .497
Revisiona x GroupA
b 102.126 2 51.063 1.004 .375 .044
Error (Revisiona) 2237.049 44 50.842
Between subjects
Intercept 108560.688 1 108560.688 978.109 .000 .957
GroupAb 499.266 2 249.633 2.249 .117 .093
Error 4883.577 44 110.990
Note. aRevision indicates the qualities of the initial and revised clinical decision.
bGroupA
indicates the three different groups— expert commentary only group (EC/NP), expert
commentary with early peer feedback group (EC/EP), and expert commentary with later peer
feedback group (EC/LP).
78
RQ2-1-2. On the quality of case assessment (EC/NP vs. EC/EP vs. EC/LP).
Hypothesis 2-1-2. There will be significant differences between the qualities of initial and
revised case assessment among the expert commentary only group (EC/NP), expert commentary
with early peer feedback group (EC/EP), and expert commentary with later peer feedback group
(EC/LP) in case-based learning.
Descriptive statistics on the quality of the initial and revised assessment in the three
groups are presented in Table 4-14. As shown in Table 4-14, the average scores of initial case
assessment of EC/NP, EC/EP, and EC/LP are 44.46 (SD = 15.79), 50.69 (SD = 14.18), and 45.69
(SD = 11.70), respectively. The average scores of revised case assessment of EC/NP, EC/NP,
EC/EP, and EC/LP are 48.07 (SD = 16.97), 57.58 (SD = 9.57), and 50.62 (SD = 16.07),
respectively.
Table 4-14
Descriptive statistics on the quality of the initial and revised case assessment among EC/NP,
EC/EP, and EC/LP
Assessment
Initial Revised
M SD M SD
EC/NPa (n=25) 44.46 15.79 48.07 16.97
EC/EPb (n=9) 50.69 14.18 57.58 9.57
EC/LPc (n=13) 45.69 11.70 50.62 16.07
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
79
Since the hypothesis 2-1 assuming significant differences between the initial and revised
clinical decisions across three groups was denied, the follow-up analysis to compare the
differences in the quality of case assessment, one of the three-dimensions, was not conducted.
RQ2-1-3. On the quality of prioritization of issues and objectives (EC/NP vs. EC/EP
vs. EC/LP). Hypothesis 2-1-3. There will be significant differences between the qualities of
initial and revised prioritization of issues and objectives among the expert commentary only
group (EC/NP), expert commentary with early peer feedback group (EC/EP), and expert
commentary with later peer feedback group (EC/LP) in case-based learning.
Descriptive statistics on the quality of the initial and revised prioritization in the three
groups are presented in Table 4-15. As shown in Table 4-15, the average scores of initial
prioritization of EC/NP, EC/EP, and EC/LP are 44.99 (SD = 11.63), 48.44 (SD = 11.40), and
48.69 (SD = 11.68), respectively. The average scores of revised case assessment of EC/NP,
EC/NP, EC/EP, and EC/LP are 48.39 (SD = 13.76), 55.08 (SD = 9.54), and 57.85 (SD = 11.93),
respectively.
Table 4-15
Descriptive statistics on the quality of the initial and revised prioritization among the EC/NP,
EC/EP, and EC/LP
Prioritization
Initial Revised
M SD M SD
EC/NPa (n=25) 44.99 11.63 48.39 13.76
EC/EPb (n=9) 48.44 11.40 55.08 9.54
EC/LPc (n=13) 48.69 11.68 57.85 11.93
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
80
Since the hypothesis 2-1 assuming significant differences between initial and revised
clinical decisions among three groups was denied, the follow-up analysis to compare the
differences in the quality of prioritization of issues and objectives, one of the three-dimensions,
was not conducted.
RQ2-1-4. On the quality of plan of an immediate action (EC/NP vs. EC/EP vs.
EC/LP). Hypothesis 2-1-4. There will be significant differences between the qualities of initial
and revised plan of an immediate action among the expert commentary only group (EC/NP),
expert commentary with early peer feedback group (EC/EP), and expert commentary with later
peer feedback group (EC/LP) in case-based learning.
Descriptive statistics on the quality of the initial and revised plan in the three groups are
presented in Table 4-16. As shown in Table 4-16, the average scores of initial plan of EC/NP,
EC/EP, and EC/LP are 46.44 (SD = 12.65), 54.81 (SD = 14.39), and 54.10 (SD = 13.83),
respectively. The average scores of revised case assessment of EC/NP, EC/EP, and EC/LP are
56.00 (SD = 14.70), 66.61 (SD = 8.64), and 65.52 (SD = 9.96), respectively.
Table 4-16
Descriptive statistics on the quality of the initial and revised plan among EC/NP, EC/EP, and
EC/LP
Plan
Initial Revised
M SD M SD
EC/NPa (n=25) 46.44 12.65 56.00 14.70
EC/EPb (n=9) 54.81 14.39 66.61 8.64
EC/LPc (n=13) 54.10 13.83 65.52 9.96
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
81
Since the hypothesis 2-1 assuming significant differences between initial and revised
clinical decisions among three groups was denied, the follow-up analysis to compare the
differences in the quality of plan of an immediate action, one of the three-dimensions, was not
conducted.
RQ2-2. Revision effects of peer feedback (EC/NP vs. EC/EP and EC/LP)
Hypothesis 2-2. There will be significant differences in the quality of students’ initial and
revised clinical decisions between groups with peer feedback (EC/EP and EC/LP) and without
peer feedback (EC/NP) in case-based learning.
Research question 2-2 examines the effects of the peer feedback on the quality of
students’ revised clinical decisions. To test the interaction effects between revision and group,
the initial and revised clinical decision responses of the participants in the peer feedback groups
and no peer feedback group were collected. A total of four different kinds of scores on the
quality of the clinical decision were calculated: scores on the quality of the three sub-dimensions
(case assessment, prioritization of issues and objectives, and plan of an immediate action) and
overall averages of the three sub-dimensions scores.
RQ2-2-1. On the overall quality of the decision (EC/NP vs. EC/EP and EC/LP).
Hypothesis 2-2-1. The students who received peer feedback (EC/EP and EC/LP) will outperform
the students who did not receive peer feedback (EC/NP) in terms of the overall quality of initial
and revised clinical decision.
Descriptive statistics on the quality of the initial and revised decisions between peer
feedback and no peer feedback groups are summarized in Table 4-17. As described in Table 4-
17, the average scores of initial decision of peer feedback and no peer feedback groups are 50.24
(SD = 11.14) and 45.30 (SD = 11.02), respectively. The average scores of revised decision of
82
peer feedback groups and no peer feedback group are 58.72 (SD = 8.14) and 50.82 (SD = 12.93),
respectively.
Table 4-17
Descriptive statistics of the overall quality of the initial and revised clinical decisions between
peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP)
Overall
Initial Revised
M SD M SD
Peer feedback groupa (n=22) 50.24 11.14 58.72 8.14
No peer feedback groupb (n=25) 45.30 11.02 50.82 12.93
Note. aPeer feedback group indicates EC/EP group (n=9) who received the expert commentary
videos as well as early peer feedback and EC/LP group (n=13) who received the expert
commentary videos as well as later peer feedback. bNo peer feedback group indicates EC group
who received the expert commentary videos only.
To examine whether the differences between the initial and revised decisions by the
treatment and control groups were significant, a repeated-measures ANOVA was conducted (see
Table 4-18). The results revealed a significant revision effect (F = 46.142, p < .05) indicating
that there were statistically significant differences between the initial and revised answers
throughout the two sessions. In other words, the overall quality of the initial answers was
significantly improved through the revision in the case-based online module. The difference
between the initial and revised answers was a large effect, with η2 = .506 (Cohen, 1988).
On the other hand, the interaction effect between revision and group was not significant
(F = 2.054, p > .05) indicating that the differences between the qualities of the initial and revised
clinical decisions were similar regardless of the groups. Thus, the hypothesis 3-1 which assumes
that the initial and revised scores of the students who were in the scaffolded group will be higher
than those of the students in the no scaffolded group was denied.
83
Table 4-18
Summary of repeated-measures ANOVA on the overall quality of the initial and revised clinical
decisions in the peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP)
Source Sum of
Squares
df Mean square F p η2
Within subjects
Revisiona 2293.816 1 2293.816 46.142 .000 .506
Revisiona x GroupB
b 102.110 1 102.110 2.054 .159 .044
Error (Revision) 2237.065 45 49.713
Between subjects
Intercept 123030.910 1 123030.910 1129.718 .000 .962
GroupBb 482.160 1 482.160 4.427 .041 .090
Error 4900.683 45 108.904
Note. aRevision indicates the quality of the initial and revised clinical decision.
bGroupB
indicates two different groups—peer feedback groups (EC/EP and EC/LP: Expert commentary
with early or later peer feedback) and no peer feedback group (EC/NP: Expert commentary
only).
RQ2-2-2. On the quality of case assessment (EC/NP vs. EC/EP and EC/LP).
Hypothesis 2-2-2. The students who received peer feedback (EC/EP and EC/LP) will outperform
the students who did not receive peer feedback (EC/NP) in terms of the quality of initial and
revised case assessment.
Descriptive statistics on the quality of the initial and revised clinical decisions in the two
groups are presented in Table 4-19. As shown in Table 4-19, the average scores of initial
decision of peer feedback groups and no peer feedback group are 47.74 (SD = 12.70) and 44.46
(SD = 15.79), respectively. The average scores of revised decision of peer feedback groups and
no peer feedback group are 53.47 (SD = 13.95) and 48.07 (SD = 16.97), respectively.
84
Table 4-19
Descriptive statistics on the quality of the initial and revised case assessment between peer
feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP)
Case Assessment
Initial Revised
M SD M SD
Peer feedback groupa (n=22) 47.74 12.70 53.47 13.95
No peer feedback groupb (n=25) 44.46 15.79 48.07 16.97
Note. aPeer feedback group indicates EC/EP group (n=9) who received the expert commentary
videos as well as early peer feedback and EC/LP group (n=13) who received the expert
commentary videos as well as later peer feedback. bNo peer feedback group indicates EC/NP
group who received the expert commentary videos only.
Since the hypothesis 3-1 assuming significant differences between initial and revised
clinical decisions by two groups was denied, the follow-up analysis to compare the differences in
the quality of case assessment, one of the three-dimensions, was not conducted.
RQ2-2-3. On the quality of prioritization of issues and objectives (EC/NP vs. EC/EP
and EC/LP). Hypothesis 2-2-3. The students who received peer feedback (EC/EP and EC/LP)
will outperform the students who did not receive peer feedback (EC/NP) in terms of the quality
of initial and revised prioritization of issues and objectives.
Descriptive statistics on the quality of the initial and revised clinical decisions in the two
groups are presented in Table 4-20. As shown in Table 4-20, the average scores of initial
decision of peer feedback groups and no peer feedback group are 48.59 (SD = 11.29) and 44.99
(SD = 11.63), respectively. The average scores of revised decision of peer feedback groups and
no peer feedback group are 56.72 (SD = 10.86) and 48.39 (SD = 13.76), respectively.
85
Table 4-20
Descriptive statistics on the quality of the initial and revised prioritization between peer
feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP)
Prioritization
Initial Revised
M SD M SD
Peer feedback groupa (n=22) 48.59 11.29 56.72 10.86
No peer feedback groupb (n=25) 44.99 11.63 48.39 13.76
Note. aPeer feedback group indicates EC/EP group (n=9) who received the expert commentary
videos as well as early peer feedback and EC/LP group (n=13) who received the expert
commentary videos as well as later peer feedback. bNo peer feedback group indicates EC/NP
group who received the expert commentary videos only.
Since the hypothesis 3-1 assuming significant differences between initial and revised
clinical decisions by two groups was denied, the follow-up analysis to compare the differences in
the quality of prioritization of issues and objectives, one of the three-dimensions, was not
conducted.
RQ2-2-4. On the quality of plan of an immediate action (EC/NP vs. EC/EP and
EC/LP). Hypothesis 2-2-4. The students who received peer feedback (EC/EP and EC/LP) will
outperform the students who did not receive peer feedback (EC/NP) in terms of the quality of
initial and revised plan of an immediate action.
Descriptive statistics on the quality of the initial and revised clinical decisions in the two
groups are presented in Table 4-21. As shown in Table 4-21, the average scores of initial
decision of peer feedback groups and no peer feedback group are 54.39 (SD = 13.72) and 46.44
(SD = 12.65), respectively. The average scores of revised decision of peer feedback groups and
no peer feedback group are 65.97 (SD = 9.24) and 56.00 (SD = 14.70), respectively.
86
Table 4-21
Descriptive statistics of the quality of the initial and revised plan between peer feedback (EC/EP
and EC/LP) and no peer feedback groups (EC/NP)
Plan
Initial Revised
M SD M SD
Peer feedback groupa (n=22) 54.39 13.72 65.97 9.24
No peer feedback groupb (n=25) 46.44 12.65 56.00 14.70
Note. aPeer feedback group indicates EC/EP group (n=9) who received the expert commentary
videos as well as early peer feedback and EC/LP group (n=13) who received the expert
commentary videos as well as later peer feedback. bNo peer feedback group indicates EC/NP
group who received the expert commentary videos only.
Since the hypothesis 3-1 assuming significant differences between initial and revised
clinical decisions by two groups was denied, the follow-up analysis to compare the differences in
the quality of prioritization of issues and objectives, one of the three-dimensions, was not
conducted.
RQ2-3. Revision effects of the timing of the peer feedback (EC/EP vs. EC/LP)
Hypothesis 2-3. There will be significant differences in the quality of students’ initial and
revised clinical decisions between the early peer feedback group (EC/EP) and later peer feedback
group (EC/LP) in case-based learning.
Research question 2-3 examines the effects of the timing of the peer feedback on the
quality of students’ revised clinical decisions. To test the interaction effects between revision
and group, the initial and revised clinical decision responses of the participants in the early and
later peer feedback groups were collected. A total of four different kinds of scores on the quality
of the clinical decision were calculated: scores on the quality of the three sub-dimensions (case
assessment, prioritization of issues and objectives, and plan of an immediate action) and overall
averages of the three sub-dimensions scores.
87
RQ2-3-1. On the overall quality of the decision (EC/EP vs. EC/LP). Hypothesis 2-3-1.
There will be significant differences in the overall quality of the initial and revised clinical
decisions between the early peer feedback group (EC/EP) and later peer feedback group (EC/LP)
in case-based learning.
Descriptive statistics on the quality of the initial and revised clinical decisions in the two
groups are presented in Table 4-22. As shown in Table 4-22, the average scores of initial
decision of EC/EP and EC/LP are 51.31 (SD = 12.47) and 49.49 (SD = 10.59), respectively. The
average scores of revised decision of EC/EP and EC/LP are 59.76 (SD = 7.42) and 57.99 (SD =
8.82), respectively.
Table 4-22
Descriptive statistics on the overall quality of the initial and revised clinical decisions between
EC/EP and EC/LP
Overall
Initial Revised
M SD M SD
EC/EPa (n=9) 51.31 12.47 59.76 7.42
EC/LPb (n=13) 49.49 10.59 57.99 8.82
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
To examine whether the differences between the initial and revised decisions between
two groups were significant, a repeated-measures ANOVA was conducted (see Table 4-23). The
results revealed a significant revision effect (F = 20.404, p < .05) indicating that there were
statistically significant differences between the initial and revised answers. In other words, the
overall quality of the initial answers was significantly improved through the revision in the case-
88
based online module. The difference between the initial and revised answers was a large effect,
with η2 = .505 (Cohen, 1988).
On the other hand, the interaction effect between revision and group was not significant
(F = .213, p > .05) indicating that the differences in the quality of the initial and revised clinical
decisions were similar across the early and later peer feedback groups. Thus, hypothesis 3-1
which assumes different quality of the initial and revised clinical decisions by groups was denied.
Table 4-23
Summary of a repeated-measures ANOVA for the overall quality of the initial and revised
clinical decisions in the EC/EP and EC/LP groups
Source Sum of
Squares
df Mean
square
F p η2
Within subjects
Revisiona 1526.926 1 1526.926 20.404 .000 .505
Revisiona x GroupC
b .016 1 .016 .000 .988 .000
Error (Revision) 1496.708 20 74.835
Between subjects
Intercept 63511.085 1 63511.085 790.001 .000 .975
GroupCb 17.106 1 17.106 .213 .650 .011
Error 1607.873 20 80.394
Note. aRevision indicates the qualities of the initial and revised clinical decision.
bGroupC
indicates the two different groups— expert commentary with early peer feedback group
(EC/EP) and expert commentary with later peer feedback group (EC/LP).
RQ2-3-2. On the quality of case assessment (EC/EP vs. EC/LP). Hypothesis 2-3-2.
There will be significant differences in the quality of the initial and revised case assessment
between the early peer feedback group (EC/EP) and later peer feedback group (EC/LP) in case-
based learning.
89
Descriptive statistics on the quality of the initial and revised clinical decisions in the two
groups are presented in Table 4-24. As shown in Table 4-24, the average scores of initial
decision of early peer feedback group and later peer feedback group are 50.69 (SD = 14.18) and
45.69 (SD = 11.70), respectively. The average scores of revised decision of early peer feedback
group and later peer feedback group are 57.58 (SD = 9.57) and 50.62 (SD = 16.07), respectively.
Table 4-24
Descriptive statistics of the qualities of the initial and revised case assessment between EC/EP
and EC/LP
Case assessment
Initial Revised
M SD M SD
EC/EPa (n=9) 50.69 14.18 57.58 9.57
EC/LPb (n=13) 45.69 11.70 50.62 16.07
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
Since the hypothesis 2-3 assuming significant differences between initial and revised
clinical decisions between two groups was denied, the follow-up analysis to compare the
differences in the quality of case assessment, one of the three-dimensions, was not conducted.
RQ2-3-3. On the quality of prioritization of issues and objectives (EC/EP vs. EC/LP).
Hypothesis 2-3-3. There will be significant differences in the quality of the initial and revised
prioritization of issues and objectives between the early peer feedback group (EC/EP) and later
peer feedback group (EC/LP) in case-based learning.
Descriptive statistics on the quality of the initial and revised clinical decisions in the two
groups are presented in Table 4-25. As shown in Table 4-25, the average scores of initial
decision of early peer feedback group and later peer feedback groups are 48.44 (SD = 11.40) and
90
48.69 (SD = 11.68), respectively. The average scores of revised decision of peer feedback group
and no peer feedback groups are 55.08 (SD = 9.54) and 57.85 (SD = 11.93), respectively.
Table 4-25
Descriptive statistics on the quality of the initial and revised prioritization between EC/EP and
EC/LP
Prioritization
Initial Revised
M SD M SD
EC/EPa (n=9) 48.44 11.40 55.08 9.54
EC/LPb (n=13) 48.69 11.68 57.85 11.93
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
Since the hypothesis 2-3 assuming significant differences between initial and revised
clinical decisions among three groups was denied, the follow-up analysis to compare the
differences in the quality of prioritization of issues and objectives, one of the three-dimensions,
was not conducted.
RQ2-3-4. On the quality of plan of an immediate action (EC/EP vs. EC/LP).
Hypothesis 2-3-4. There will be significant differences in the quality of the initial and revised
plan of an immediate action between the early peer feedback group (EC/EP) and later peer
feedback group (EC/LP) in case-based learning.
Descriptive statistics on the quality of the initial and revised clinical decisions in the two
groups are presented in Table 4-26. As shown in Table 4-26, the average scores of initial
decision of EC/EP and EC/LP are 54.81 (SD = 14.39) and 54.10 (SD = 13.83), respectively. The
average scores of revised decision of EC/EP and EC/LP are 66.61 (SD = 8.64) and 65.52 (SD =
9.96), respectively.
91
Table 4-26
Descriptive statistics of the qualities of the initial and revised plan between expert commentary
with early peer feedback group (EC/EP) and expert commentary with later peer feedback group
(EC/LP)
Plan
Initial Revised
M SD M SD
EC/EPa (n=9) 54.81 14.39 66.61 8.64
EC/LPb (n=13) 54.10 13.83 65.52 9.96
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
Since the hypothesis 2-3 assuming significant differences between initial and revised
clinical decisions among three groups was denied, the follow-up analysis to compare the
differences in the quality of plan of an immediate action, one of the three-dimensions, was not
conducted.
RQ3. Revision effects by groups across sessions
Research Question 3. Are there significant differences in the quality of the initial and
revised clinical decision among groups across two sessions?
o Research Question 3-1. Are there significant differences in the quality of students’
initial and revised clinical decision among EC/NP (expert commentary only),
EC/EP (expert commentary and early peer feedback), and EC/LP (expert
commentary and later peer feedback) across two sessions?
o Research Question 3-2. Are there significant differences in the quality of students’
initial and revised clinical decision between peer feedback groups (EC/EP and
EC/LP) and no peer feedback group (EC/NP) across two sessions?
92
o Research Question 3-3. Are there significant differences in the quality of students’
initial and revised clinical decision between the early peer feedback group (EC/EP)
and later peer feedback group (EC/LP) across two sessions?
The third research question examines whether there is any significant difference in the
quality of the initial and revised clinical decisions among groups across two sessions. To test the
three-way interaction effect among revision, group and session, and three different group
variations across two sessions were tested.
Table 4-27
Data used to test the three-way interaction effect (Revision x group x session) (Research
Question 3)
Session 1 Session 2 Across Sessions 1 & 2 Session 3
Initial Revised Initial Revised Initial Revised Transfer
test
EC/NPa NP1-1 NP1-2 NP2-1 NP2-2 - - -
EC/EPb EP1-1 EP1-2 EP2-1 EP2-2 - - -
EC/LPc LP1-1 LP1-2 LP2-1 LP2-2 - - -
Total - - - - - -
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
RQ3-1. Revision effects among groups across sessions (EC/NP vs. EC/EP vs. EC/LP)
Hypothesis 3-1. There will be significant differences in the quality of students’ initial and
revised clinical decision among the expert commentary only group (EC/NP), expert commentary
with early peer feedback group (EC/EP), and expert commentary with later peer feedback group
(EC/LP) across two sessions.
93
In order to test the hypothesis, the initial and revised clinical decision responses of all
participants were collected. A total of four different kinds of scores on the quality of the clinical
decision were calculated: scores on the quality of the three sub-dimensions (case assessment,
prioritization of issues and objectives, and plan of an immediate action) and overall averages of
the three sub-dimensions scores.
RQ3-1-1. On the overall quality of the decision across sessions (EC/NP vs. EC/EP vs.
EC/LP). Hypothesis 3-1-1. There will be significant differences in the overall quality of the
initial and revised clinical decision among the expert commentary only group (EC/NP), expert
commentary with early peer feedback group (EC/EP), and expert commentary with later peer
feedback group (EC/LP) across two sessions.
Descriptive statistics on the quality of the initial and revised clinical decisions in the
EC/NP, EC/EP, and EC/LP across the two sessions are presented in Table 4-28.
Table 4-28
Descriptive statistics of the overall quality of the initial and revised clinical decisions among the
EC/NP, EC/EP, and EC/LP across two sessions
Overall
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
EC/NPa (n=25) 39.79 15.18 45.68 16.33 50.81 9.52 55.96 11.64
EC/EPb (n=9) 47.50 13.95 61.04 11.14 55.13 15.43 58.48 13.03
EC/LPc (n=13) 38.72 15.53 49.58 14.07 60.27 12.36 66.41 7.78
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
94
To examine whether the differences between the initial and revised decisions by three
groups were significant, a repeated-measures ANOVA with two within variables (the initial and
revised decision-making responses and two sessions) and one between variable (group) was
conducted. Also, the interaction effects for revision by group effects, revision by session, and
revision by session by group effects were considered to test the hypothesis.
The results showed that there were no statistically significant effects for the revision and
group interaction (F = 1.004, p > .05) and the three-way interaction among the revision, session,
and group (F = 1.781, p > .05). This result indicates that there was no significant difference
between the three groups’ overall quality of the initial and revised clinical decisions across the
two sessions.
95
Table 4-29
Summary of a repeated-measures ANOVA for the overall quality of the initial and revised
clinical decisions among the EC/NP, EC/EP, and EC/LP
Source Sum of
Squares
df Mean
square
F p η2
Within subjects
Revisiona 4427.415 1 4427.415 43.541 .000 .497
Revisiona x GroupA
b 204.253 2 102.126 1.004 .375 .044
Error (Revisiona) 4474.098 44 101.684
Sessionc 2298.826 1 2298.826 27.574 .000 .385
Sessionc x GroupA
b 755.114 2 377.557 4.529 .016 .171
Error (Session) 3668.295 44 83.370
Revisiona x Session
c 536.559 1 536.559 6.257 .016 .124
Revisiona x Session
c x GroupA
b 305.453 2 152.727 1.781 .180 .075
Error (Revisiona x Session
c) 3773.386 44 85.759
Between subjects
Intercept 217121.376 1 217121.376 978.109 .000 .957
GroupAb 998.532 2 499.266 2.249 .117 .093
Error 9767.155 44 221.981
Note. aRevision indicates the initial and revised decision-making scores.
bGroupA indicates the
three different groups— expert commentary only group (EC/NP), expert commentary with
early peer feedback group (EC/EP), and expert commentary with later peer feedback group
(EC/LP). cSession indicates the two sessions comprising of two Decision Points—The first
session was comprised of DP2 and 3, and the second session was comprised of DP4 and 5.
RQ3-1-2. On the quality of case assessment across sessions (EC/NP vs. EC/EP vs.
EC/LP). Hypothesis 3-1-2. There will be significant differences in the quality of the initial and
revised case assessment among the expert commentary only group (EC/NP), expert commentary
96
with early peer feedback group (EC/EP), and expert commentary with later peer feedback group
(EC/LP) across two sessions.
Descriptive statistics on the quality of the initial and revised assessment in the EC/NP,
EC/EP, and EC/LP are presented in Table 4-30. Since the hypothesis 3-1 assuming significant
differences between initial and revised clinical decisions among the three groups was denied, the
follow-up analysis to compare the differences in the quality of case assessment, one of the three-
dimensions, was not conducted.
Table 4-30
Descriptive statistics of the overall quality of the initial and revised case assessment among
EC/NP, EC/EP, and EC/LP across two sessions
Assessment
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
EC/NPa (n=25) 31.82 21.52 36.70 22.26 57.10 15.28 59.44 16.56
EC/EPb (n=9) 40.11 17.59 52.83 19.45 61.28 15.89 62.33 14.86
EC/LPc (n=13) 28.04 18.17 37.08 26.44 63.35 12.49 64.15 12.75
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
RQ3-1-3. On the quality of prioritization of issues and objectives across sessions
(EC/NP vs. EC/EP vs. EC/LP). Hypothesis 3-1-3. There will be significant differences in the
quality of the initial and revised prioritization among the expert commentary only group
(EC/NP), expert commentary with early peer feedback group (EC/EP), and expert commentary
with later peer feedback group (EC/LP) across two sessions.
97
Descriptive statistics on the quality of the initial and revised assessment in the EC/NP,
EC/EP, and EC/LP are presented in Table 4-31. Since the hypothesis 3-1 assuming significant
differences between initial and revised clinical decisions among the three groups was denied, the
follow-up analysis to compare the differences in the quality of prioritization of issues and
objectives, one of the three-dimensions, was not conducted.
Table 4-31
Descriptive statistics of the overall quality of the initial and revised prioritization of issues and
objectives among the EC/NP, EC/EP, and EC/LP across two sessions
Prioritization
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
EC/NPa (n=25) 32.96 16.68 37.12 19.45 57.02 13.88 59.66 13.08
EC/EPb (n=9) 37.50 13.71 53.44 9.62 59.39 16.46 56.72 14.39
EC/LPc (n=13) 36.00 19.66 48.39 18.68 61.39 11.91 67.31 11.70
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
RQ3-1-4. On the quality of plan of an immediate action across sessions (EC/NP vs.
EC/EP vs. EC/LP). Hypothesis 3-1-4. There will be significant differences in the quality of the
initial and revised plan among the expert commentary only group (EC/NP), expert commentary
with early peer feedback group (EC/EP), and expert commentary with later peer feedback group
(EC/LP) across two sessions.
Descriptive statistics of the quality of the initial and revised assessment in the EC/NP,
EC/EP, and EC/LP are presented in Table 4-32. Since the hypothesis 3-1 assuming significant
differences between initial and revised clinical decisions among the three groups was denied, the
98
follow-up analysis to compare the differences in the quality of plan of an immediate action, one
of the three-dimensions, was not conducted.
Table 4-32
Descriptive statistics of the overall quality of the initial and revised plan of an immediate action
among EC/NP, EC/EP, and EC/LP across two sessions
Prioritization
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
EC/NPa (n=25) 54.58 19.81 63.22 17.39 38.30 15.35 48.78 17.46
EC/EPb (n=9) 64.89 21.34 76.83 12.60 44.72 20.53 56.39 17.46
EC/LPc (n=13) 52.12 18.18 63.27 16.48 56.08 21.33 67.77 17.45
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
RQ3-2. Revision effects of peer feedback across sessions (EC/NP vs. EC/EP and EC/LP)
Hypothesis 3-2. There will be significant differences in the quality of students’ initial and
revised clinical decisions between peer feedback groups (EC/EP and EC/LP) and no peer
feedback group (EC/NP) across sessions.
In order to test the hypothesis, the initial and revised clinical decision responses of all
participants were collected. A total of four different kinds of scores on the quality of the clinical
decision were calculated: scores on the quality of the three sub-dimensions (case assessment,
prioritization of issues and objectives, and plan of an immediate action) and overall averages of
the three sub-dimensions scores.
RQ3-2-1. On the overall quality of the decision across sessions (EC/NP vs. EC/EP
and EC/LP). Hypothesis 3-2-1. The students who received peer feedback (EC/EP and EC/LP)
99
will outperform the students who did not receive peer feedback (EC/NP) in terms of the overall
quality of initial and revised clinical decision across two sessions.
Descriptive statistics on the quality of the initial and revised decisions between peer
feedback groups (EC/EP and EC/LP) and no peer feedback groups (EC/NP) across sessions are
summarized in Table 4-33.
Table 4-33
Descriptive statistics of the overall qualities of the initial and revised clinical decisions between
peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP) across two sessions
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
Peer feedback
groupa (n=22)
42.31 15.21 54.27 13.92 58.17 13.59 63.17 10.73
No peer feedback
groupb (n=25)
39.79 15.18 45.68 16.33 50.81 9.52 55.96 11.64
Note. aPeer feedback group indicates EC/EP group (n=9) who received the expert commentary
videos as well as early peer feedback and EC/LP group (n=13) who received the expert
commentary videos as well as later peer feedback. bNo peer feedback group indicates EC/NP
group who received the expert commentary videos only.
To examine whether the differences between the initial and revised decisions by the peer
feedback and no peer feedback groups were significant, a repeated-measures ANOVA with two
within variables (the initial and revised decision-making responses and two sessions) and one
between variable (group) was conducted (see Table 4-34). Also, the interaction effects for
revision by group effects, revision by session, and revision by session by group effects were
considered to test the hypothesis.
The results showed that there were no statistically significant effects for the revision and
group interaction (F = 2.054, p > .05) and the three-way interaction among the revision, session,
100
and group (F = 2.639, p > .05). This result indicates that there was no significant difference
between the peer feedback and no peer feedback groups in the overall quality of the initial and
revised clinical decisions across the two sessions.
Table 4-34
Summary of repeated-measures ANOVA for the quality of the initial and revised case assessment
in the peer feedback and no peer feedback groups
Source Sum of
Squares
df Mean
square
F p η2
Within subjects
Revisiona 4587.631 1 4587.631 46.142 .000 .506
Revisiona x GroupB
b 204.220 1 204.220 2.054 .159 .044
Error (Revisiona) 4474.131 45 99.425
Sessionc 3102.966 1 3102.966 31.692 .000 .413
Sessionc x GroupB
b 17.487 1 17.487 .179 .675 .004
Error (Session) 4405.922 45 97.909
Revisiona x Session
c 346.418 1 346.418 4.046 .050 .082
Revisiona x Session
c x GroupB
b 225.971 1 225.971 2.639 .111 .055
Error (Revisiona x Session
c) 3852.868 45 85.619
Between subjects
Intercept 246061.820 1 246061.820 1129.718 .000 .962
GroupAb 964.320 1 964.320 4.427 .041 .090
Error 9801.366 45 217.808
Note. aRevision indicates the initial and revised decision-making scores.
bGroupB indicates
two different groups—peer feedback groups (EC/EP and EC/LP: Expert commentary with early
or later peer feedback) and no peer feedback group (EC/NP: Expert commentary only). cSession indicates the two sessions comprising of two Decision Points—The first session was
comprised of DP2 and 3, and the second session was comprised of DP4 and 5.
101
RQ3-2-2. On the quality of case assessment across sessions (EC/NP vs. EC/EP and
EC/LP). Hypothesis 3-2-2. The students who received peer feedback (EC/EP and EC/LP) will
outperform the students who did not receive peer feedback (EC/NP) in terms of the quality of
initial and revised case assessment across two sessions.
Descriptive statistics on the quality of the initial and revised case assessment between
peer feedback groups (EC/EP and EC/LP) and no peer feedback groups (EC/NP) across sessions
are summarized in Table 4-35. Since the hypothesis 3-2 assuming significant differences
between initial and revised clinical decisions between peer feedback and no peer feedback
groups was denied, the follow-up analysis to compare the differences in the quality of case
assessment, one of the three-dimensions, was not conducted.
Table 4-35
Descriptive statistics of the quality of the initial and revised case assessment between peer
feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP) across two sessions
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
Peer feedback
groupa (n=22)
32.98 18.53 43.52 24.63 62.50 13.66 63.41 13.34
No peer feedback
groupb (n=25)
31.82 21.52 36.70 22.26 57.10 15.28 59.44 16.56
Note. aPeer feedback group indicates EC/EP group (n=9) who received the expert commentary
videos as well as early peer feedback and EC/LP group (n=13) who received the expert
commentary videos as well as later peer feedback. bNo peer feedback group indicates EC/NP
group who received the expert commentary videos only.
RQ3-2-3. On the quality of prioritization of issues and objectives across sessions
(EC/NP vs. EC/EP and EC/LP). Hypothesis 3-2-3. The students who received peer feedback
(EC/EP and EC/LP) will outperform the students who did not receive peer feedback (EC/NP) in
102
terms of the quality of initial and revised prioritization of issues and objectives across two
sessions.
Descriptive statistics on the quality of the initial and revised prioritization between peer
feedback groups (EC/EP and EC/LP) and no peer feedback groups (EC/NP) across sessions are
summarized in Table 4-36. Since the hypothesis 3-2 assuming significant differences between
initial and revised clinical decisions between peer feedback and no peer feedback groups was
denied, the follow-up analysis to compare the differences in the quality of prioritization of issues
and objectives, one of the three-dimensions, was not conducted.
Table 4-36
Descriptive statistics on the quality of the initial and revised prioritization of issues and
objectives between peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC/NP)
across two sessions
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
Peer feedback
groupa (n=22)
36.61 17.12 50.46 15.53 60.57 13.62 62.98 13.62
No peer feedback
groupb (n=25)
32.96 16.68 37.12 19.45 57.02 13.88 59.66 13.08
Note. aPeer feedback group indicates EC/EP group (n=9) who received the expert commentary
videos as well as early peer feedback and EC/LP group (n=13) who received the expert
commentary videos as well as later peer feedback. bNo peer feedback group indicates EC/NP
group who received the expert commentary videos only.
RQ3-2-4. On the quality of plan of an immediate action across sessions (EC/NP vs.
EC/EP and EC/LP). Hypothesis 2-2-4. The students who received peer feedback (EC/EP and
EC/LP) will outperform the students who did not receive peer feedback (EC/NP) in terms of the
quality of initial and revised plan of an immediate action across two sessions.
103
Descriptive statistics on the quality of the initial and revised plan between peer feedback
groups (EC/EP and EC/LP) and no peer feedback groups (EC/NP) across sessions are
summarized in Table 4-37. Since the hypothesis 3-2 assuming significant differences between
initial and revised clinical decisions between peer feedback and no peer feedback groups was
denied, the follow-up analysis to compare the differences in the quality of plan of an immediate
action, one of the three-dimensions, was not conducted.
Table 4-37
Descriptive statistics on the quality of the initial and revised plan of an immediate action
between peer feedback (EC/EP and EC/LP) and no peer feedback groups (EC) across two
sessions
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
Peer feedback
groupa (n=22)
57.34 20.09 68.82 16.19 51.43 21.29 63.11 17.97
No peer feedback
groupb (n=25)
54.58 19.81 63.22 17.39 38.3 15.35 48.78 17.46
Note. aPeer feedback group indicates EC/EP group (n=9) who received the expert commentary
videos as well as early peer feedback and EC/LP group (n=13) who received the expert
commentary videos as well as later peer feedback. bNo peer feedback group indicates EC/NP
group who received the expert commentary videos only.
RQ3-3. Revision effects of timing of the peer feedback across sessions (EC/EP vs. EC/LP)
Hypothesis 3-3. There will be significant differences in the quality of students’ initial and
revised clinical decisions between groups with early peer feedback (EC/EP) and with later peer
feedback (EC/LP) across sessions.
In order to test the hypothesis, the initial and revised clinical decision responses of all
participants in the EC/EP and EC/LP were collected. A total of four different kinds of scores on
104
the quality of the clinical decision were calculated: scores on the quality of the three sub-
dimensions (case assessment, prioritization of issues and objectives, and plan of an immediate
action) and overall averages of the three sub-dimensions scores.
RQ3-3-1. On the overall quality of the decision across sessions (EC/EP vs. EC/LP).
Hypothesis 3-3-1. There will be significant differences in the overall quality of the initial and
revised clinical decisions between the early peer feedback group (EC/EP) and later peer feedback
group (EC/LP) in case-based learning across sessions.
Descriptive statistics on the quality of the initial and revised clinical decisions between
EC/EP and EC/LP across sessions are summarized in Table 4-38.
Table 4-38
Descriptive statistics on the overall quality of the initial and revised clinical decisions between
EC/EP and EC/LP across two sessions
Overall
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
EC/EPa (n=9) 47.50 13.95 61.04 11.14 55.13 15.43 58.48 13.03
EC/LPb (n=13) 38.72 15.53 49.58 14.07 60.27 12.36 66.41 7.78
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
To examine whether the differences between the initial and revised decisions by EC/EP
and EC/LP were significant across sessions, a repeated-measures ANOVA with two within
variables (the initial and revised decision-making responses and two sessions) and one between
variable (group) was conducted (see Table 4-39). Also, the interaction effects for revision by
105
group effects, revision by session, and revision by session by group effects were considered to
test the hypothesis.
The results showed that there were no statistically significant effects for the revision and
group interaction (F = .000, p > .05) and the three-way interaction among the revision, session,
and group (F = .674, p > .05). This result indicates that there was no significant difference in the
overall quality of the initial and revised clinical decisions between EC/EP and EC/LP across the
two sessions.
106
Table 4-39
Summary of repeated-measures ANOVA for the quality of the initial and revised case assessment
in EC/EP and EC/LP
Source Sum of
Squares
df Mean
square
F p η2
Within subjects
Revisiona 3053.851 1 3053.851 20.404 .000 .505
Revisiona x GroupC
b .033 1 .033 .000 .988 .000
Error (Revisiona) 2993.417 20 149.671
Sessionc 1255.528 1 1255.528 11.080 .003 .357
Sessionc x GroupC
b 737.627 1 737.627 6.510 .019 .246
Error (Session) 2266.288 20 113.314
Revisiona x Session
c 590.593 1 590.593 5.006 .037 .200
Revisiona x Session
c x GroupC
b 79.482 1 79.482 .674 .421 .033
Error (Revisiona x Session
c) 2359.356 20 117.968
Between subjects
Intercept 127022.171 1 127022.171 790.001 .000 .975
GroupAb 34.211 1 34.211 .213 .650 .011
Error 3215.745 20 160.787
Note. aRevision indicates the initial and revised decision-making scores
bGroupC indicates the
two different groups— expert commentary with early peer feedback group (EC/EP) and expert
commentary with later peer feedback group (EC/LP). cSession indicates the two sessions
comprising of two Decision Points—The first session was comprised of DP2 and 3, and the
second session was comprised of DP4 and 5.
RQ3-3-2. On the quality of case assessment across sessions (EC/EP vs. EC/LP).
Hypothesis 3-3-2. There will be significant differences in the quality of the initial and revised
107
case assessment between the early peer feedback group (EC/EP) and later peer feedback group
(EC/LP) across sessions.
Descriptive statistics on the quality of the initial and revised assessment between early
peer feedback group (EC/EP) and later peer feedback groups (EC/LP) across sessions are
summarized in Table 4-40. Since the hypothesis 3-3 assuming significant differences between
initial and revised clinical decisions between early peer feedback group and later peer feedback
group was denied, the follow-up analysis to compare the differences in the quality of case
assessment, one of the three-dimensions, was not conducted.
Table 4-40
Descriptive statistics on the quality of the initial and revised case assessment between early peer
feedback group and later peer feedback group across two sessions
Assessment
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
EC/EPa (n=9) 40.11 17.59 52.83 19.45 61.28 15.89 62.33 14.86
EC/LPb (n=13) 28.04 18.17 37.08 26.44 63.35 12.49 64.15 12.75
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
RQ3-3-3. On the quality of prioritization of issues and objectives across sessions
(EC/EP vs. EC/LP). Hypothesis 2-3-3. There will be significant differences in the quality of the
initial and revised prioritization of issues and objectives between the early peer feedback group
(EC/EP) and later peer feedback group (EC/LP) across sessions.
Descriptive statistics on the quality of the initial and revised assessment between early
peer feedback group (EC/EP) and later peer feedback groups (EC/LP) across sessions are
108
summarized in Table 4-40. Since the hypothesis 3-3 assuming significant differences between
initial and revised clinical decisions between early peer feedback group and later peer feedback
group was denied, the follow-up analysis to compare the differences in the quality of
prioritization of issues and objectives, one of the three-dimensions, was not conducted.
Table 4-41
Descriptive statistics of the quality of the initial and revised prioritization of issues and
objectives between early peer feedback group and later peer feedback group across two sessions
Prioritization
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
EC/EPa (n=9) 37.50 13.71 53.44 9.62 59.39 16.46 56.72 14.39
EC/LPb (n=13) 36.00 19.66 48.39 18.68 61.39 11.91 67.31 11.70
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
RQ3-3-4. On the quality of plan of an immediate action across sessions (EC/EP vs.
EC/LP). Hypothesis 2-3-4. There will be significant differences in the quality of the initial and
revised plan of an immediate action between the early peer feedback group (EC/EP) and later
peer feedback group (EC/LP) in case-based learning across sessions.
Descriptive statistics on the quality of the initial and revised assessment between early
peer feedback group (EC/EP) and later peer feedback groups (EC/LP) across sessions are
summarized in Table 4-40. Since the hypothesis 3-3 assuming significant differences between
initial and revised clinical decisions between early peer feedback group and later peer feedback
group was denied, the follow-up analysis to compare the differences in the quality of plan of an
immediate action, one of the three-dimensions, was not conducted.
109
Table 4-42
Descriptive statistics on the quality of the initial and revised plan of an immediate action
between early peer feedback group and later peer feedback group across two sessions
Plan
Session 1 Session 2
Initial Revised Initial Revised
M SD M SD M SD M SD
EC/EPa (n=9) 64.89 21.34 76.83 12.60 44.72 20.53 56.39 17.46
EC/LPb (n=13) 52.12 18.18 63.27 16.48 56.08 21.33 67.77 17.45
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
RQ4. Transfer effects
Research Question 4. Does the participation in the scaffolded revision activities affect
students’ transferred clinical decision-making skills?
o Research Question 4-1. Are there significant differences in the scores on a
transferred clinical decision test among EC/NP (expert commentary only), EC/EP
(expert commentary and early peer feedback), and EC/LP (expert commentary
and later peer feedback)?
o Research Question 4-2. Are there significant differences in the scores on a
transferred clinical decision test between groups with peer feedback (EC/EP and
EC/LP) and without peer feedback (EC/NP)?
o Research Question 4-3. Are there significant differences in the scores on a
transferred clinical decision test between groups with early peer feedback
(EC/EP) and with later peer feedback (EC/LP)?
110
The fourth research question examines whether the scaffolded revision activities (expert
commentary and/or peer feedback) affects students’ transferred clinical decision-making skills.
To test students’ transferred clinical decision-making skills, 6 multiple-choice clinical decision-
making questions were developed and distributed upon the completion of the case-based online
learning module. The transferred test scores of all participants from the three groups were
collected.
Table 4-43
Data used to test the transferred effects of the scaffolded revision activities (Research Question 4)
Session 1 Session 2 Across Sessions 1 & 2 Session 3
Initial Revised Initial Revised Initial Revised Transfer
test
EC/NPa - - - - - - EC-3
EC/EPb - - - - - - EP-3
EC/LPc - - - - - - LP-3
Total - - - - - -
Note. aEC indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
RQ4-1. Transferred effect by groups (EC/NP vs. EC/EP and EC/LP)
Hypothesis 4-1. There will be significant differences in the scores on a transferred
clinical decision test among the expert commentary only group (EC/NP), expert commentary
with early peer feedback group (EC/EP), and expert commentary with later peer feedback group
(EC/LP).
In order to test the hypothesis 4-1, all participants’ performances in the transferred
clinical decision test were collected. Descriptive statistics on the scores on the transfer test from
111
the EC/NP, EC/EP, and EC/LP are summarized in Table 4-44. The highest score of the transfer
test was 6.
Table 4-44
Descriptive statistics on the scores on the transfer test among EC/NP, EC/EP, and EC/LP
Group
N
Transfer Test
M SD
EC/NPa 24 5.29 0.62
EC/EPb 9 5.56 0.53
EC/LPc 11 4.91 1.04
Total 44 5.25 0.75
Note. aEC/NP indicates the group who received the expert commentary videos only.
bEC/EP
indicates the group who received the expert commentary videos as well as early peer feedback. cEC/LP indicates the group who received the expert commentary videos as well as later peer
feedback.
To examine whether there is significant difference between three groups, a one-way
ANOVA was conducted. The results showed that there was no statistically significant effect for
the performance on the transfer test among EC/NP, EC/EP, and EC/LP (F = 2.005, p > .05).
112
Table 4-45
Summary of One-way ANOVA for the scores on the transfer test EC/NP, EC/EP, and EC/LP
Source Sum of
Squares
df Mean
square
F p η2
Corrected Model 2.160a 2 1.080 2.005 .148 .089
Intercept 1018.772 1 1018.772 1890.916 .000 .979
GroupAb 2.160 2 1.080 2.005 .148 .089
Error 22.090 41 .539
Total 1237.000 44
Corrected Total 24.250 43
Note. aR Squared = .089 (Adjusted R Squared = .045).
bGroupA indicates the three different
groups— expert commentary only group (EC/NP), expert commentary with early peer feedback
group (EC/EP), and expert commentary with later peer feedback group (EC/LP).
RQ4-2. Transferred effect of the peer feedback (EC/NP vs. EC/EP and EC/LP)
Hypothesis 4-2. There will be significant differences in the scores on a transferred
clinical decision test between peer feedback groups (EC/EP and EC/LP) and no peer feedback
group (EC/NP).
Descriptive statistics on the scores on the transfer test from the participants in the peer
feedback groups (EC/EP and EC/LP) and no peer feedback group (EC/NP) are summarized in
Table 4-46. The highest score of the transfer test was 6.
113
Table 4-46
Descriptive statistics of the scores on the transfer test between the peer feedback groups (EC/EP
and EC/LP) and no peer feedback group (EC/NP)
Group
N
Transfer Test
M SD
Expert Commentary only (EC) 24 5.29 0.62
Expert Commentary with Peer feedback (EC/EP & EC/LP) 20 5.20 0.89
Total 44 5.25 0.75
To examine whether there is significant difference between two groups, one-way
ANOVA was conducted. The results showed that there was no statistically significant effect for
the performance on the transfer test between groups with peer feedback (EC/EP and EC/LP) and
without peer feedback (EC/NP) (F = .159, p > .05).
Table 4-47
Summary of One-way ANOVA for the scores on the transfer test between groups with peer
feedback (EC/EP and EC/LP) and without peer feedback (EC/NP)
Source Sum of Squares df Mean square F p η2
Corrected Model .092a 1 .092 .159 .692 .004
Intercept 1200.819 1 1200.819 2087.660 .000 .980
GroupBb .092 1 .092 .159 .692 .004
Error 24.158 42 .575
Total 1237.000 44
Corrected Total 24.250 43
Note. aR Squared = .004 (Adjusted R Squared = -.020).
bGroupB indicates two different
groups—peer feedback groups (EC/EP and EC/LP: Expert commentary with early or later peer
feedback) and no peer feedback group (EC/NP: Expert commentary only).
114
RQ4-3. Transferred effects of the timing of the peer feedback (EC/EP vs. EC/LP)
Hypothesis 4-3. There will be significant differences in the scores on a transferred
clinical decision test between early peer feedback group (EC/EP) and later peer feedback group
(EC/LP).
Descriptive statistics on the scores on the transfer test from the participants in the early
peer feedback groups (EC/EP) and later peer feedback group (EC/LP) are summarized in Table
4-48. The highest score of the transfer test was 6.
Table 4-48
Descriptive statistics of the scores on the transfer test between EC/EP and EC/LP
Group
N
Transfer Test
M SD
EC/EPa 9 5.56 0.53
EC/LPb 11 4.91 1.04
Total 44 5.20 0.89
Note. aEC/EP indicates the group who received the expert commentary videos as well as early
peer feedback. bEC/LP indicates the group who received the expert commentary videos as well
as later peer feedback.
To examine whether there is significant difference between two groups, an one-way
ANOVA was conducted. The results showed that there was no statistically significant effect for
the performance on the transfer test between the early peer feedback (EC/EP) and the later peer
feedback groups (EC/LP) (F = .159, p > .05).
115
Table 4-49
Summary of One-way ANOVA for the scores on the transfer test between groups with early peer
feedback (EC/EP) and with later peer feedback (EC/LP)
Source Sum of
Squares
df Mean
square
F p η2
Corrected Model 2.069a 1 2.069 2.836 .109 .136
Intercept 542.069 1 542.069 743.051 .000 .976
GroupCb 2.069 1 2.069 2.836 .109 .136
Error 13.131 18 .730
Total 556.000 20
Corrected Total 15.200 19
Note. aR Squared = .136 (Adjusted R Squared = .088).
b bGroupC indicates the two different
groups— expert commentary with early peer feedback group (EC/EP) and expert commentary
with later peer feedback group (EC/LP).
RQ5. Student perception on the revision experiences
Research Question 5. What are the students’ perceptions on the revision activity in the
case-based online learning module?
o Research Question 5-1. What are the students’ perceptions on the expert
commentary for revising their initial clinical decisions?
o Research Question 5-2. What are the students’ perceptions on the peer feedback
for revising their initial clinical decisions?
o Research Question 5-3. What are the students’ perceptions on the effectiveness of
the peer feedback compared to the expert commentary?
In order to collect data on how students participated in the scaffolded revision activity
(expert commentary and/or peer feedback session) and how they valued their experience with the
scaffolded revision activity, an online survey and face-to-face interviews were conducted.
116
As the online survey was embedded in the closing Decision Point, the students who
finished all Decision Points were able to answer the survey. The survey was conducted
anonymously to gain more honest responses from the participants. Respondents were asked to
select the scaffolded revision activity they participated in: expert commentary only group (EC),
expert commentary with early peer feedback group (EC/EP), or expert commentary with later
peer feedback group (EC/LP). The participants in EC group were asked to express their
perceptions of the learning experiences with expert commentary. The participants in either
EC/EP group or EC/LP group were asked to express their perceptions of the learning experiences
with expert commentary as well as with peer feedback. They were also asked to compare the
helpfulness of the scaffolded revision activity between the expert commentary and peer feedback.
Face-to-face interviews were conducted in December, upon the completion of the module.
Since the purpose of the interviews was to explore students’ learning experiences with peer
feedback in detail, interviewees were recruited from either EC/EP group or EC/LP group only.
A total of three female interviewees volunteered to share their learning experiences: one was
from the EC/EP group, and the other two interviewees were from the EC/LP group.
RQ5-1. Student perception on the expert commentary
Survey results. To examine the participants’ perceptions of the scaffolded revision
activity with expert commentary, two multiple-choice questions and the follow-up open-ended
essay questions were asked. The brief results of the two multiple-choice questions are
summarized in Table 4-50. As described in the table, the participants showed positive attitudes
toward the scaffolded revision activity with expert commentary, with an average of 4.46.
117
Table 4-50
Means and standard deviations of the items about student perceptions on the scaffolded-revision
activity with expert commentary
Item
EC/NPa
(n=80)
EC/EPb
(n=9)
EC/LPc
(n=13)
Total
M (SD) M (SD) M (SD) M (SD)
1. I think the selfrevision activity was
helpful.
4.34
(0.67)
4.33
(0.50)
4.23
(0.73)
4.32
(0.66)
2. I think the selfrevision activity helped me
reflect on the gaps between experts’ opinions
and mine.
4.64
(0.56)
4.44
(0.53)
4.46
(0.52)
4.60
(0.55)
Overall 4.49 4.38 4.35 4.46
Note. aEC/NP indicates the group who received expert commentary videos only.
bEC/EP
indicates the group who received expert commentary videos with early peer feedback. cEC/LP
indicates the group who received expert commentary videos with later peer feedback.
On the follow-up question asking the reason why they thought the scaffolded revision
activity with expert commentary was helpful, especially in reflecting on the gaps between their
opinions and those of the experts, some participants reported that it was good to hear from
experts before revising their answers. It seemed that the video enabled the participants not only
“to still state [their] own thought process,” but also “to make sure [they] had all the facts straight
in [their] logic.” Some participants responded that the expert commentary videos enabled them
to “solidify points that [they] learned in class,” “realize what details [they] were missing related
to the lesson,” “reinforce it by adding it to [their] decision,” and “see [the experts’] clinical
application.” The participants also thought the comparison between their opinions and those of
the experts was helpful to “give [them] reassurance that [they were] on the right track.” With
this matter, a participant responded, “it made me more confident in my answers if there was a
question I was unsure about, or to go back and revise if there was something I didn't understand
118
correctly.” Furthermore, they thought the expert commentary videos helped them “identify areas
in [their] thought process that [they] may have potentially left out” and “understand how [their]
reasoning needs to be improved.”
RQ5-2. Student perception on the peer feedback
Survey results. To examine the participants’ perceptions of the scaffolded revision
activity with peer feedback, five multiple-choice questions and their follow-up open-ended essay
questions were distributed. The brief results of the five multiple-choice questions are
summarized in Table 4-51. As described in the table, the participants showed positive responses
toward the peer feedback session, with an average of 3.72. In particular, the group who received
later peer feedback (EC/LP) was more positive (M = 4.03) than the group who received early
peer feedback (EC/EP) (M = 3.27).
119
Table 4-51
Means and standard deviations of the items about student perceptions on the scaffolded revision
with peer feedback
Item EC/EPa
(n=9)
EC/LPb
(n=13)
Total
M (SD) M (SD) M (SD)
1. I think the peer feedback activity was
helpful.
3.11 (1.17) 4.15 (0.56) 3.73 (0.99)
2. I think the peer feedback activity helped me
reflect on the gaps between experts’ opinions
and mine.
2.89 (1.27) 3.85 (0.80) 3.45 (1.10)
3. I think the question prompts were helpful. 3.33 (1.00) 3.62 (1.12) 3.50 (1.06)
4. I think providing feedback was
helpful/meaningful.
3.44 (1.13) 4.23 (0.60) 3.91 (0.92)
5. I think receiving feedback was
helpful/meaningful.
3.56 (1.13) 4.31 (0.63) 4.00 (0.93)
Total 3.27 4.03 3.72
Note. aEC/EP indicates the group who received expert commentary videos with early peer
feedback. bEC/LP indicates the group who received expert commentary videos with later peer
feedback.
On the follow-up questions asking the reason why they thought the peer feedback session
was helpful, especially in reflecting on the gaps between their opinions and those of the experts,
the participants reported that listening to other classmates’ opinions on the same problem was
helpful. In particular, the participants valued “different ideas and new ways of looking at
things.” For example, a participant reported “[my] peers can help explain their interpretations of
the experts opinions in a different but very relatable way.” Also, they mentioned that discussing
their responses with peers enabled them to “gain confidence in [their] answers, and communicate
and solidify [their] thoughts relative to these cases” as well as “fill in [their] knowledge gaps.”
120
However, several participants thought that the additional involvement of the experts in
the peer feedback session would have been more beneficial. They felt discussions with peers
only had some limitations. For example, they reported that “there were some points [they] still
[weren’t] clear on,” and peers were not helpful, because they “all interpreted the questions
differently.” Also, some participants addressed they had difficulties in focusing on the module,
“because everyone else was around.” A participant expressed two hours were not enough to
complete the two Decision Points.
On the question asking the effectiveness of the provided question prompts, some
participants reported that the question prompts were helpful, especially in leading them “as to
what [the experts] wanted [them] to discuss” as well as in helping them “organize information.”
They believed that the prompts were “an accurate representation of certain clinical decisions that
one will have to make in practice.”
However, some participants did not find the reflective prompts were helpful, because
they were too general and redundant (e.g., “They were fairly repetitive or maybe too general, and
I found myself answering repetitively.”). Also, some participants mentioned they never used the
prompts and “just discussed [their] rationales and read over each other's work to make sure it
read the way [they] intended it to.”
On the question of the effectiveness of providing and/or receiving feedback, the
respondents did not differentiate the effectiveness between providing and receiving feedback.
Instead, they expressed their general attitudes toward having a feedback session among peers.
Some respondents showed positive attitudes toward peer feedback: they reported “discussing the
case with others helps [them] learn and retain information better.” Specifically, they believed
peer feedback helped them “explain [their] opinion,” “give a rationale for [their] choice,”
121
“reconsider some of [their] points and logic,” “correct anything that wasn’t clear to other people,”
“refine communication skills,” and “gain confidence in [their] answers.”
Other respondents, on the other hand, reported that the peer feedback “really didn’t
change much” for them. They valued the expert opinions over the peer feedback in that “[they]
had the expert opinions so [they] knew to an extent the "right answer" before [they] were getting
or giving any peer feedback.” Due to this reason, they “didn't result in anyone changing their
answers really.”
To improve the peer feedback activity, several suggestions were made by the participants.
First, many respondents wished that “it would have been nice to get a different expert opinion on
certain aspects.” Specifically, they said “a little more structure or open conversation led by a
teacher during the feedback sessions,” such as “allowing clinicians to at least take questions and
choose whether or not to answer” would have been better. Second, having a bigger-sized group
discussion was suggested. They thought “a whole table discussion instead of paired off” would
allow more discussion. Third, a participant suggested that “[having] peer feedback BEFORE
listening to the experts” would have been better. The participant said “[it] would be more
helpful if instead of watching the videos and then discussing, [they] discuss the case itself with
clinicians and [their] peers, similar to rounds.”
Interview results. First, they were asked to describe the reason why they decided to
voluntarily participate in the peer feedback session. Interviewee A participated in the session,
because “it helps [her] work through case better.” She said, “[e]ven though I could go through it
faster by myself, I feel like sometimes I don’t retain information well when I do that. So I can
talk to someone else hear their ideas too, it helps me retain things better.” Another reason why
she chose to participate in the session was that she wanted to see how other students study with
122
the module. Interviewees B and C participated in the session, because they thought there would
be a chance to ask questions to the experts.
The peer review session wasn’t quite what I thought it was going to be. I guess [the
instructor] was more involved. She was going to bring up important points. I thought she
was going to guide us as the experts did in the videos, which doesn’t really make sense
because that’s why the expert videos are there. So she doesn’t have to do that. But I
thought she was going to stimulate discussions. It was more read someone else’s
assessment and point out for them what the differences (Interviewee C, Lines 268-273).
On the follow-up question on activities they expected before attending the session, all
three participants mentioned that they expected interactions with the experts to some extent.
They seemed to expect that the experts would “help direct [their] thinking” and answer questions
that some participants might have. For example, interviewee A responded she might want to
“talk with the professors about medical decisions, maybe that would’ve given [her] more
direction in what to put.” Interviewee B mentioned that she might have asked experts the most
perplexing questions, like case assessment—selecting two or three most important cues out of
five or six cues.
The participants then were asked to describe how they provide and receive feedback with
their partner. Interviewee A said she and her partner mostly talked about the patient and the
medical case itself. Interviewee B responded that she showed her responses to her partner and
the partner provided feedback accordingly. However, both respondents mentioned that they did
not provide or receive feedback much. Interviewee C and her partner also showed their own
123
responses and compared each other’s responses. She said the partner’s responses were helpful
for reminding her of the points she missed. While exchanging the responses, she wrote notes for
her and added the points she missed during the revision time. She said, however, her initial
responses would have been much better if she had enough time to write. She said, “[I] was
trying to type so fast,” and she forgot to write down although she thought of it.
We talked about like if the question had to do with, what we would, what the goals were
for the patient, just hearing what they had to say about the goals, they thought they were
important and I contribute what I thought, which goals I thought were important. If they
had a good argument for one of their goals that I hadn’t written down, if I agree with
them after discussing it then I would include that goal. Or maybe if I had a goal the same
as one they had but maybe if they had expanded on the information a little more on their
answer, then I would think about it more and include the more information about it
(Interview A, Line 116-122).
We swapped and had our partner read ours and give us suggestions. Initially, I think
throughout the entire thing, mine were pretty long descriptions as you can see. My
partner’s were very like short. So when she was reading mine, she was like “I don’t
really have many things for you to add or change, because you wrote so much.” She did
make a couple of suggestions, but… So for the revising, I didn’t change that much. I
might’ve changed a sentence or two. I didn’t change very much (Interviewee B, Line
272-277).
124
I was thinking like, “Oh, man. I knew. I thought of this. I was trying to type so fast, and
I forgot to write it down. I thought of it, but I forgot to write it down.” Then when we
come to the revision section, I would like to try to add the things I’ve forgotten. I wrote
notes for myself while we were correcting each other. “I forgot to write this. I forgot to
write this.” And I went back to add it during the revision time (Interviewee C, Lines 308-
313).
The interviewees thought they did not have much feedback to provide each other,
because they have “the same mindset and the same thought processes” by experiencing the same
courses together. For this reason, the interviewees suggested it would have been helpful if they
had a chance to receive feedback from experts who “might have whole different side point of
view than [them] who are just focused on thinking about what [they]’ve learned in this course
and how that relates.”
On the question whether they revised much during the peer feedback session, interviewee
A said she revised her initial decision much based on her peer’s feedback and her insights while
exchanging feedback with her partner. On the other hand, interviewee B did not revise very
much. She said, “I wanted to say and then after I listened to the experts, I was like, I addressed
all the same things that they addressed.” She also did not identify herself as a group studier, so
she was not able to concentrate on the module as much as she did the module alone at home.
She also thought her partner was not focused either, so “[their] suggestions [were] a little bit
superficial.” She also said, “we were might have been not as honest as what we thought that
would change them what should happened.”
125
On the follow-up question asking their opinions about anonymous peer feedback session,
interviewee B and C showed positive reactions to it, because they thought people would give
more honest and deeper feedback.
When you were sitting with someone and talking about it, you want to agree with what
they wrote ‘cause you don’t want them to feel bad because what they wrote was valid. I
feel like definitely you don’t want someone else to feel bad about what you’re critiquing
them. So, yes, I do think that had it been anonymous thing, you could’ve addressed on
your own time, you would’ve get more feedback (Interviewee C, Lines 351-355).
On the question of comparing the value of providing or receiving feedback for each other,
interviewee B found providing feedback more valuable. She addressed that receiving feedback
did not help her think about her own opinion during that time, while providing feedback inspired
her to think about how she would change her partner’s responses, which eventually resulted in
reflecting back on to her own responses.
RQ5-3. Student perception on the helpfulness of the scaffolded revision activities
Survey results. To examine the participants’ perceptions on the effectiveness of the
expert commentary and peer feedback, three open-ended essay questions were used. On the
question which scaffolded activity was more helpful for them to revise their answers, eight
participants out of nine in the early peer feedback group (EC/EP) preferred the expert
commentary videos to peer feedback, whereas nine participants out of 13 in the later peer
feedback session (EC/LP) preferred peer feedback to the expert commentary only.
126
The participants, who preferred the expert commentary only, responded that the expert
commentary videos were enough for them to gain insights how to form, reinforce or change their
own decisions. Some participants even mentioned, “The peer feedback really didn’t bring up
any points I hadn't thought about or that hadn't been addressed by the expert opinions.” In
particular, the participants believed that receiving feedback from peers with the same level of
training as them was not helpful. Several participants said, “At this point in our career we have
all taken the same courses and formed very similar opinions based on that, but did not consider
everything that the experts did when it came to making decisions,” “When I had questions or
comments none of us knew the answers.” In addition to this, several participants expressed that
they were more focused when studying alone rather than studying with peers. They seemed to
“like to take time to think, search through books and research to try to come up with the best
answer.” In the peer review, on the other hand, several participants seemed to rush into
answering everything.
The participants who preferred peer feedback activity to the expert commentary only
mentioned that they were able to read their own responses more closely with their peers. They
also found the peer feedback activity meaningful in that they were able to get out of their
comfort zone and see the same case from a different perspective (e.g., “I liked the peer feedback
more since I only have a limited perspective and limited number of ideas when I talk to myself.
Talking to another people helps me see new things.”). Also, they valued the discussion with
peers who were in the same level of training in that “it allowed for discussion.” “[The peer
feedback activity] helped to assist me in the thought process behind answering the questions
about the case. The self-revision activity was helpful in correcting my mistakes during my initial
interpretation of the case, but that was better for learning the correct material as opposed to the
127
process of reaching that correct answer.” Last, the participants thought the peer feedback
activity allowed them to “come prepared and confident and get more out of the session.”
Interview results. The three participants in the interviews were asked to compare their
learning experiences between the peer feedback activity and those in the self-revision activity.
Interviewee A participated in the first peer feedback session, and interviewee B and C
participated in the second peer feedback session.
They were asked to compare the effectiveness of the peer feedback activity and the self-
revision activity. Interviewee A preferred peer feedback activity, and interviewees B and C
preferred self-revision activity. Interviewee A said, “Even though it went faster when I was
doing by myself, because I had to stop to talk to someone else (in the peer feedback session), I
feel like, seeing how someone else thinks about a problem versus how I do was helpful.”
Interviewees B and C found the self-revision more helpful to them. They both mentioned
that they were more focused when no one was around. Although interviewee B said, “I guess the
peer session might’ve been easier to do, because I had other people’s opinions too. It gave me a
little bit more confident or something. We both had the same thought processes. That kind of
thing made me feel easier,” she added, “It was definitely much more helpful sit at home and do
this on my own, look up the resources that I need to look up, not talk to anybody about it.”
Interviewee C also valued the self-revision activity over the peer feedback activity. She seemed
to be less coherent in her writing during the peer feedback session, because she was rushing
herself into writing to keep pace with her partner. She said, “If I was doing by myself, I
would’ve been thinking more thoroughly what I was saying and writing sentences full sense.
These probably make sense, but I feel like I was probably writing so quickly that I might’ve
missed the point. I might’ve not made the point.”
128
When the interviewees were asked to grade their revised answers in the peer feedback
session and those in the self-revision activity, interviewee A expected that the quality of the
revised responses in the peer feedback session was higher, while interviewee B and C expected
the revised responses in the self-revision activity were better. Interviewee A thought, “because I
had more information from, to include from my partner and maybe different things I forgot from
class to include in there.” Interviewees B and C, on the other hand, expected that they were
doing better in the self-revision activity, because they were able to have more in-depth and
thorough thinking.
Then, they were asked to rank the helpfulness of the expert commentary videos and peer
feedback session in enhancing their revision. All three interviewees believed that the expert
commentary videos were most helpful in facilitating their revision. Interviewee A put the peer
feedback activity close to the expert videos “because getting to talk to other people helps it to
cement things in [her] brain better,” while other interviewees ranked the peer feedback activity
as not helpful, because they did not identify themselves as a “group studier.”
129
CHAPTER 5
CONCLUSION
In this study, the assumption was made that promoting veterinary students’ knowledge
application and reflection would enhance their clinical decision-making skills. The case-based
online learning module was developed in order to promote the students’ knowledge application,
and the scaffolded revision activities were designed to promote their reflection on their thinking
and actions.
With the case-based online learning module and scaffolded revision activities, the
students in this study were expected to make a series of clinical decisions (referred to as initial
clinical decision) and then revise the decisions (referred to as revised clinical decision). In the
initial decision-making activity, the students were asked to watch the case videos, identify and
analyze the problems, and make a decision with an aid of critical thinking prompts. In the
revision of the decision, they received a chance to compare their opinions to those of experts or
of peers. The expert commentary videos narrating expert veterinarians’ decision-making
approaches and their own decisions were provided to all participants. In addition, peer feedback
was provided in two separate sessions, and the students were allowed to participate in either or
none of the sessions. Thus, all participants were allowed to participate in one of the three diverse
scaffolded revision activities: expert commentary with no peer feedback (EC/NP), expert
commentary with early peer feedback (EC/EP), or expert commentary with later peer feedback
(EC/LP). This study examined whether the quality of the students’ revised decisions was
130
significantly enhanced after participating in one of the scaffolded revision activities and
identified which scaffolded revision activity was most helpful.
A brief summary of the results and discussions related to the results are described and
explained according to research questions in this chapter. Also implications of the study and
suggestions for future research are also discussed.
Summary of the Findings
This research explored five research questions to examine the effects of the scaffolded
revision activities on veterinary students’ clinical decision-making skills. Specifically, the first
three research questions examined the gain effects of the revision activities, the fourth research
question examined the transfer effects of the revision activities, and the last research question
explored the students’ perception on the scaffolded revision activities.
Research Question 1. Gain Effect—Revision Effect
The first research question tested whether the quality of the students’ revised clinical
decisions was significantly enhanced after experiencing the scaffolded revision activities. The
results indicated that the qualitative changes between the initial and the revised clinical decisions
were statistically significant within the three sub-dimensions of case assessment, prioritization of
issues and objectives, and plan of an immediate action.
Research Question 2. Gain Effect—Revision x Group Effect
After verifying the significant differences in the quality of the initial and revised clinical
decisions, the second research question tested the quality of the initial and revised clinical
decisions among the groups based on the three diverse scaffolded revision activities. The first
comparison was between each group of EC/NP, EC/EP, and EC/LP being compared to one
another on the basis of the quality of the initial and revised clinical decisions. The second
131
comparison was between the EC/NP group to the other two groups, EC/EP and EC/LP, with the
addition of peer feedback. Lastly, the third comparison is between the two peer feedback groups,
EC/EP and EC/LP, to each other to see how the timing of the peer feedback affected the quality
of the initial and revised clinical decisions.
RQ2-1. Significant enhancement among EC/NP, EC/EP, and EC/LP. RQ2-1 tested if
there is a statistically significant difference in the quality of the initial and revised clinical
decisions among the groups with expert commentary only (EC/NP), with expert commentary and
early peer feedback (EC/EP), and with expert commentary and later peer feedback (EC/LP). The
results indicated that there was no statistically significant difference among the groups.
RQ2-2. Significant enhancement between peer feedback group (EC/EP and EC/LP)
vs. no peer feedback group (EC/NP). RQ2-2 tested if there is a significant difference in the
quality of the initial and revised clinical decisions between the groups with peer feedback
(EC/EP and EC/LP) and the group without peer feedback (EC/NP). The results indicated that
indicated that the differences between two groups were not significant.
RQ2-3. Significant enhancement between early peer feedback (EC/EP) and later
peer feedback (EC/LP). RQ2-3 tested if there is a significant difference in the quality of the
initial and revised clinical decisions between the group with early peer feedback (EC/EP) and the
group with later peer feedback (EC/LP). The results indicated that the differences between two
groups were not significant.
Research Question 3. Gain Effect—Revision x Group x Session Effect
The third research question examined whether or not there were any significant
differences in the quality of the initial and revised clinical decisions among the groups when
viewed and compared in two separate sessions. Similar to the second research question, the first
132
comparison is between each group of EC/NP, EC/EP, and EC/LP being compared to one another
on the basis of the quality of the initial and revised clinical decisions. Next, the second
comparison is between the EC/NP group to the other two groups, EC/EP and EC/LP, with the
addition of peer feedback. Then the third comparison is between the two peer feedback groups,
EC/EP and EC/LP, to each other to see how the timing of the peer feedback affected the quality
of the initial and revised clinical decisions. Finally, each of the above three comparison sets are
compared across two separate sessions.
RQ3-1. Significant enhancement among EC/NP, EC/EP, and EC/LP across sessions.
RQ3-1 tested if there is a significant difference in the quality of the initial and revised clinical
decisions among the groups with expert commentary only (EC/NP), with expert commentary and
early peer feedback (EC/EP), and with expert commentary and later peer feedback (EC/LP)
across the two sessions. The results indicated that the differences among groups across the
sessions were not significant.
RQ3-2. Significant enhancement between peer feedback group (EC/EP and EC/LP)
vs. no peer feedback group (EC/NP) across sessions. RQ3-2 tested if there is a significant
difference in the quality of the initial and revised clinical decisions between the groups with peer
feedback (EC/EP and EC/LP) and the group without peer feedback (EC/NP) across the two
sessions. The results indicated that the differences between groups across the sessions were not
significant.
RQ3-3. Significant enhancement between early peer feedback (EC/EP) and later
peer feedback (EC/LP) across sessions. RQ3-3 tested if there is a significant difference in the
quality of the initial and revised clinical decisions between the group with early peer feedback
133
(EC/EP) and the group with later peer feedback (EC/LP) across the two sessions. The results of
indicated the differences between groups across the sessions were not significant.
Research Question 4. Transfer Effect
The fourth research question examined whether there was any significant difference in
the students’ performance on the transfer test. The transfer test comprised of six multiple-choice
questions with text-based cases of digestive diseases and patients’ medical records. The students
were asked to interpret data, identify the problems, and make a clinical decision for the
patient. In order to test the transfer effect of the scaffolded revision activities, three comparison
sets were identified. The first comparison is between each group of EC/NP, EC/EP, and EC/LP
being compared to one another on the basis of the performance on the transfer test. Next, the
second comparison is between the EC/NP group to the other two groups, EC/EP and EC/LP,
with the addition of peer feedback. Then the third comparison is between the two peer feedback
groups, EC/EP and EC/LP, to each other to see how the timing of the peer feedback affected the
students’ performance on the transfer test.
RQ4-1. Transfer effects among EC/NP, EC/EP, and EC/LP. RQ4-1 tested if there is a
significant difference in the performance on the transfer test among the groups with expert
commentary only (EC), with expert commentary and early peer feedback (EC/EP), and with
expert commentary and later peer feedback (EC/LP) across the two sessions. The results
indicated that the difference among group was not significant.
RQ4-2. Transfer effects between peer feedback group (EC/EP and EC/LP) vs. no
peer feedback group (EC/NP). RQ4-2 tested if there is a significant difference in the
performance on the transfer test between the groups with peer feedback (EC/EP and EC/LP) and
134
the group without peer feedback (EC/NP). The results indicated that the difference between the
two groups was not significant.
RQ4-3. Transfer effects between early peer feedback (EC/EP) and later peer
feedback (EC/LP). RQ4-3 tested if there is a significant difference in the performance on the
transfer test between the groups with early peer feedback (EC/EP) and the group with later peer
feedback (EC/LP). The results indicated that the difference between the two groups was not
significant.
Research Question 5. Students’ Perception
The fifth research question examined the students’ perception on the revision activity
using expert commentary and peer feedback. To explore the students’ perception, an online
survey and face-to-face interviews were conducted.
RQ5-1. Perception on the expert commentary videos. The participant’s perception on
the scaffolded revision activity with the expert commentary videos was positive in general. The
participants mentioned that the expert videos solidify what they have learned in class and helped
them learn how experts use the knowledge in real practices. Also, they acknowledged that
comparing their opinions and those of experts was a great opportunity for them to check whether
their clinical decision-making process is correct. Moreover, the participants believed that the
expert commentary was meaningful for them to identify their weak areas in their thought process
that need to be improved.
RQ5-2. Perception on the peer feedback. The participants’ perception on the
scaffolded revision activity with peer feedback was positive in general. The participants, in
particular, believed that peer feedback allowed them to experience different but very relatable
interpretations on the experts’ opinions. Also, comparing their own opinions to those of
135
colleagues was helpful to retain information and fill in the knowledge gaps. Also, they
mentioned that they were able to reconsider some of their points, and thus, better explain and
give a rationale for their choices, which resulted in gaining confidence and refining
communication skills.
To some participants, however, peer feedback did not help their revision
activity. Because these peers share similar levels of knowledge based on what they have learned
together in similar courses, these individuals rarely had differing opinions. For this reason,
several participants expressed that it would have been more beneficial to them if the experts had
also been involved in the peer feedback sessions and guided their discussions, specifically on
what to discuss as well as how to discuss.
RQ5-3. Comparison between expert commentary only and with peer feedback.
Concerning the perception of which revision activity was more helpful, the majority of the
participants in the EC/EP group who received early peer feedback preferred the expert
commentary only, whereas the majority of the participants in the EC/LP group who received
later peer feedback preferred the expert commentary with peer feedback.
The participants who preferred the revision activity with expert commentary only felt that
the expert commentary videos were enough to promote their revision. They explained that their
peers might have limitations in providing meaningful feedback to stimulate revision. They also
mentioned that they had difficulties in focusing on the learning module due to having other
people physically there, which contrasted sharply with the individualistic atmosphere of an
online learning environment. The participants who preferred the revision activity with expert
commentary and peer feedback, on the other hand, believed that discussions with peers provoked
deeper critical thinking and honed their communication skills.
136
Effects of the case-based online learning module and scaffolded revision
Veterinarians’ clinical decision-making skills can be improved upon when they engage in
critically thinking about how to apply their academic knowledge and subsequently, make an
informed decision. The results of the current study showed that learning with the case-based
online module having students revisit and revise their decision-making processes has the
potential to improve their clinical decision-making skills. This section discusses the
interpretation of the effects of the expert commentary as well as the effects of peer feedback.
Then the rest of the section discusses the effects between the expert commentary and peer
feedback as well as early peer feedback to later peer feedback.
Effects of the Expert Commentary
The quantitative results of this study showed that the expert commentary could enhance
the students’ clinical decision-making skills within the three sub-areas of case assessment,
prioritization of issues and objectives, and plan of an immediate action. Consistent with previous
research (e.g., Croskerry & Nimmo, 2011; Gielen, Peeters, Dochy, Onghena, & Struyven, 2010;
Gielen, Tops, Dochy, Onghena, & Smeets, 2010; Mory, 2004; Pedersen & Liu, 2002), expert
commentary could increase the accuracy of the decision. For example, in a study by Pedersen
and Liu (2002), expert modeling impacted the quality of students’ reasoning and their rationale
for their decision. In order to make a rationale clinical decision, the students in this study were
supposed to develop proficiency in utilizing knowledge they have learned in classes, making a
decision, and revising the decision throughout their work in the case-based learning module. In
this way, the expert commentary might provide the students with reliable feedback which
enables them to appreciate, understand, and correct errors, which, in turn, results in better
decision making (Croskerry & Nimmo, 2011).
137
In addition, the qualitative results from the students’ online survey and face-to-face
interviews provided more insights in what ways students could possibly benefit from the expert
commentary videos. For example, participants in this study thought the expert commentary
videos helped them solidify what they had learned in class and guided them in learning how the
acquired knowledge from school could be utilized to solve real problems. To elaborate, this
study showed that expert commentary could benefit students by helping them acquire and apply
knowledge similar to previous research (e.g., Pedersen & Liu, 2002).
Some participants also felt that the expert commentary videos facilitated further
reflection upon their decision-making process as well as provided a check upon whether their
clinical decisions were on the correct track. This benefit is consistent with previous findings that
feedback, especially when it comes from experts, can play a role as a standard of performance,
which allows learners to compare their actual performance (Butler & Winne, 1995; Mory, 2003,
2004; Winne & Hadwin, 1998).
Additionally, participants mentioned that the expert commentary videos allowed them to
identify weak areas in their decision-making reasoning. This result may indicate that expert
opinions, as external feedback, could state explicitly whether the decisions and performance of
the learner were adequate as well as model appropriate decisions (Butler & Winne, 1995; Mory,
2003, 2004; Winne & Hadwin, 1998). Also, the result of this study may support that external
feedback could provide individuals with opportunities to broaden and deepen their perspective
(van den Boom, Paas, & van Merriënboer, 2007).
Effects of the Peer Feedback
The quantitative results of this study failed to support the hypothesis that the combined
use of peer feedback and expert commentary would be more effective in enhancing clinical
138
decision-making skills than the independent use of expert commentary only. From the online
survey and the face-to-face interviews, however, participants felt that peer feedback helped them
retain knowledge better (Johnson & Johnson, 1993; Mory, 2004; Van Lehn et al., 1995) by
allowing them to communicate their thoughts with peers (Fischer, Kollar, Stegmann & Wecker,
2013; Kolodner, 2007; Sessa et al., 2011).
The effects of peer feedback were found to be not statistically effective, which seems to
be derived from the fact that the actual peer feedback sessions were different from the ideal
situation. An ideal situation for peer feedback must have students compare their ideas, exchange
constructive and suggestive feedback, and use as well as evaluate evidence (van der Pol, van den
Berg, Admiraal, & Simons, 2008). Specifically in this study, the participants mentioned that
they did not provide nor receive constructive and suggestive feedback from peers. The lack of
constructive and suggestive peer feedback could stem from learner characteristics and their
familiarity to the task. Additionally, the small sample size might explain why the results were
not statistically significant.
In terms of learner characteristics, the participants’ lack of knowledge and inexperience
would have hindered their ability to provide constructive and suggestive peer feedback. Due to
the participants’ lack of knowledge and inexperience within the veterinary field, the accuracy of
the feedback provided by peers could be lower than that of the feedback by experts (Gielen, Tops,
et al., 2010). Furthermore, the participants limited domain knowledge might cause them to be
unfamiliar with asking productive questions or elaborating on ideas and thoughts (Ge & Land,
2003). Understandably, the peer feedback within this study was less direct and concrete than if it
had been under ideal situations (Cromley & Azevedo, 2005; van den Boom et al., 2007).
139
In terms of familiarity to the task, the participants found the given task predictable due to
the fact that the case-based learning module dealt with a typical canine digestive disease, which
the participants had already learned. To elaborate, when the context of a problem is familiar to a
student, the possible course of actions tend to be relatively predictable within their decision-
making process (Shin, Jonassen, & McGee, 2003). This familiarity might discourage students
from regulating their decision-making process—planning, monitoring, and reflecting—which is
thought to be a strong predictor of successful problem solving (Shin et al., 2003).
In terms of the small sample size, it is statistically proven that the size of the sample
affects the significance of a test statistic (Eng, 2003; Field, 2013). As the descriptive results
indicated, there was a trend that the EC/EP group who received expert commentary as well as
early peer feedback had higher scores on the quality of the decisions than the other groups. Thus,
increasing the sample size might increase the statistical power that would result in significant
interaction effects. To elaborate, it is recommended to have at least 20 to 30 observations in
each cell in ANOVA/MANOVA to obtain robust statistical results (Geweke & Singleton, 1980;
Lilliefors, 1967; Machiko R. Tomita, 2006; Ntoumanis & Myers, 2016; Swanson & Holton,
2005).
Effects of the Timing of the Peer Feedback
The combined use of peer feedback and expert commentary was not statistically more
effective in enhancing clinical decision-making skills than the independent use of expert
commentary only. Additionally, the quantitative data of this study indicated that the differences
between the participants in the early peer feedback group and those in the later peer feedback
group were not statistically significant. Thus, there was no effect upon having the participants
exchange feedback in either early or later peer feedback sessions.
140
Implications of the Study
Designing learning activities that enhance students’ clinical decision-making skills
requires a thoughtful approach. This study may offer possibilities for teaching clinical decision-
making skills by promoting students’ knowledge application and reflection. In the following
section, the implications of the scaffolded revision activities in the case-based online learning
module for knowledge application and reflection are discussed.
Implications for Enhancing Knowledge Application
The results of this study showed that revision activities in case-based learning could have
the potential to bridge the gaps between theory and practice. This study may suggest two
implications that educators and instructional designers need to consider to promote students’
knowledge application: providing authentic cases and an entire cycle of decision-making.
This study may support the importance of providing authentic cases, which include real-
world challenges (Choi et al., 2013). Authentic cases with real-world challenges are believed to
lessen the gaps between theories and reality by providing an opportunity for students to
determine what resources and information they need, how to use them, and how to perform a
given task using them (J. S. Brown et al., 1989). To bolster the realism of the clinical cases, one
promising way could be the involvement of multiple players in the decision-making process
(Higgs & Jones, 2008; Orasanu & Connolly, 1993; Smith et al., 2008; Terry & Higgs,
1993). The current learning module, for example, described the owner’s financial concerns,
which is one of the characteristics that frequently influence a doctor’s decision-making
(Vandeweerd et al., 2012b).
This study may also support the importance of providing an entire cycle of decision-
making process, which is believed to help students acquire a broad range of problem-solving
141
skills (Williams, 1992). Many studies have focused on a specific phase of an entire cycle of
decision-making, such as diagnostic decision (e.g., Croskerry & Nimmo, 2011; Cutrer et al.,
2013; Elstein, Schwartz, & Schwarz, 2002). The current study, however, provided a case-based
online learning module that allowed students to experience an entire cycle of clinical decision-
making, including diagnostic, therapeutic, and prognostic decision-making.
Implications for Promoting Reflection
Consistent with other studies (e.g., Croskerry & Nimmo, 2011; Epstein, 1999; Jones,
1992; Mamede & Schmidt, 2004, 2005), the findings of this study may support the importance of
reflection in enhancing students’ clinical decision-making skills. In this study, the participants
received opportunities to make their own decisions and revise their initial decisions. In other
words, the results may indicate that encouraging students to step back from their problem
situation and reflect on their thinking process is effective in enhancing the quality of the clinical
decisions.
In addition, the findings of this study may support previous studies that have shown that
having students compare their own opinions with those of their peers enhances their clinical
decision-making skills (e.g., Croskerry & Nimmo, 2011; Gielen, Peeters, Dochy, Onghena, &
Struyven, 2010; Gielen, Tops, et al., 2010). As indicated in other previous studies (Anderson,
1987; Pirolli & Anderson, 1985), being exposed to experts’ decision-making process can benefit
students, since most students approach problem solving by referring to known examples or
developing abstract declarative rules that guide their problem solving. The participants in this
study compared their opinions with those of experts and/or those of their peers, and the results
might indicate that expert commentary has the potential to improve their clinical decision-
making skills.
142
Suggestions for future research
This study focused on the use of scaffolded revision activities in case-based learning with
third-year veterinary students to enhance their clinical decision-making skills. The results of this
study suggest several important directions for future research on case-based learning, peer
feedback, and clinical decision-making.
Testing with unfamiliar cases
This study presented the enhanced clinical decision-making skills with familiar cases.
The familiarity to the case might reduce students’ perceived difficulty level of the case, and thus,
undermine the importance of the scaffold revision activities for clinical students (Pedersen & Liu,
2002). Therefore, future studies may consider using unfamiliar cases to test the effects of case-
based learning on improving veterinary students’ clinical decision-making performance.
Providing sufficient scaffoldings to guide interactions
Consistent with previous research (Ge & Land, 2003), this study suggests that simply
providing reflective prompts might be insufficient for successfully guiding a peer feedback
session. According to the peer feedback groups’ anecdotal data, participants did not use the
reflective prompts productively. Thus, further studies may consider additional strategies, such as
instructor’s monitoring or guidance, to scaffold a peer feedback session. To elaborate, the
instructor can help students with asking their peers questions, elaborating or explaining their
thoughts, constructing arguments, or providing constructive and suggestive feedback (Ge &
Land, 2003).
Testing the generalizability of the results of this study
The participants in this study had a chance to self-select to join one of the three types of
scaffolded revision activities—EC/NP, EC/EP, or EC/LP—which resulted in unequal
143
distribution with small group sizes. Thus, it is necessary to replicate the same study with a
random sample assignment and an increased sample size to ensure the generalizability of the
results. Further research is needed to confirm the generalizability of the results.
Testing on the transferred clinical decision-making skills
This study tested both gain effects and transfer effects of the scaffolded revision
activities. However, the term between the gain tests and the transfer test was approximately one
or two weeks. From an educational standpoint, this one- to two-week gap may not be strong
enough to show the training effect of the scaffolded revision activities. Moreover, clinical
decision-making skills are rather long-term skills. In order to test the far-transferred effects of
the scaffolded revision activities and the case-based learning module, future studies may
consider using a longer gap period between interventions and transfer tests to see whether or not
an individual is indeed able to make a better clinical decision.
144
REFERENCES
Anene, B. M. (2013). Clinical decision making in veterinary practice. Nigerian Veterinary
Journal, 34(4), 877–882. Retrieved from http://etheses.nottingham.ac.uk/2051/
Banning, M. (2007). A review of clinical decision making: Models and current research. Journal
of Clinical Nursing, 17(2), 187–195. doi:10.1111/j.1365-2702.2006.01791.x
Barrows, H. S., & Tamblyn, R. M. (1980). Problem-based learning: An approach to medical
education. New York: Springer.
Borleffs, J. C. C., Custers, E. J. F. M., van Gijn, J., & ten Cate, O. T. J. (2003). “Clinical
reasoning theater”: a new approach to clinical reasoning education. Academic Medicineine,
78(3), 322–325. doi:10.1097/00001888-200303000-00017
Boud, D., & Walker, D. (1998). Promoting reflection in professional courses: The challenge of
context. Studies in Higher Education, 23(2), 191–206.
doi:10.1080/03075079812331380384
Brown, A. L. (1987). Metacognition, executive control, self-regulation, and other more
mysterious mechanisms. In F. E. Weinert & R. H. Kluwe (Eds.), Metacognition, motivation,
and understan (pp. 65–115). HUlsdale, NJ: Lawrence Erlbaum Associate.
Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning.
Educational Researcher, 18(1), 32–42.
Cameron, S., & Turtle-song, I. (2002). Learning to write case notes using the SOAP format.
Journal of Counseling & Development, 80(3), 286–292. doi:10.1002/j.1556-
6678.2002.tb00193.x
145
Campanella, M., & Lygo-Baker, S. (2014). Reconsidering the lecture in modern veterinary
education. J Vet Med Educ, 41(2), 138–45.
Chaiken, S., & Trope, Y. (1999). Dual-process theories in social psychology. New York:
Guilford Press.
Choi, I. (2009). A case-based e-learning framework for real-world problem solving : implications
for human resources development. Journal of Korean HRD Research, 4(1), 81–100.
Choi, I., Hong, Y.-C., Park, H., & Lee, Y. (2013). Case-based learning for anesthesiology:
Enhancing dynamic decision-making skills through cognitive apprenticeship and cognitive
flexibility. In R. Luckin, S. Puntambekar, P. Goodyear, B. Grabowski, J. Underwood, & N.
Winters (Eds.), Handbook of design in educational technology (pp. 230–240). New York,
NY: Routledge.
Choi, I., & Lee, K. (2009). Designing and implementing a case-based learning environment for
enhancing ill-structured problem solving: Classroom management problems for prospective
teachers. Educational Technology Research and Development, 57, 99–129.
doi:10.1007/s11423-008-9089-2
Choi, I., Lee, S. J., & Kang, J. (2009). Implementing a case-based e-learning environment in a
lecture-oriented anaesthesiology class: Do learning styles matter in complex problem
solving over time? British Journal of Educational Technology, 40(5), 933–947.
doi:10.1111/j.1467-8535.2008.00884.x
Choo, S. S. Y., Rotgans, J. I., Yew, E. H. J., & Schmidt, H. G. (2011). Effect of worksheet
scaffolds on student learning in problem-based learning. Advances in Health Sciences
Education, 16, 517–528. doi:10.1007/s10459-011-9288-1
Cockcroft, P. D. (2007). Clinical reasoning and decision analysis. Veterinary Clinics of North
146
America - Small Animal Practice, 37(3), 499–520. doi:10.1016/j.cvsm.2007.01.011
Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New Jersey: Lawrence
Erlbaum Associates Inc. Publishers.
Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking
visible. American Educator, 15(3), 6 – 11, 38–46.
Collins, A., Brown, J. S., & Newman, S. E. (1987). Cognitive apprenticeship: Teaching the craft
of reading, writing, and mathematics. Champain, Illinois.
Collyer, S. C., & Malecki, G. S. (1998). Tactical decision making under stress: History and
overview. In J. A. Cannon-Bowers & E. Salas (Eds.), Making decisions under stress:
Implications for individual and team training (pp. 3–15). Washington, DC: American
Psychological Association. doi:10.1080/01402389908425338
Cromley, J. G., & Azevedo, R. (2005). What do reading tutors do? A naturalistic study of more
and less experienced tutors in reading. Discourse Processes, 40(2), 83–113.
doi:10.1207/s15326950dp4002
Croskerry, P. (2009). A universal model of diagnostic reasoning. Academic Medicine : Journal
of the Association of American Medical Colleges, 84(8), 1022–1028.
doi:10.1097/ACM.0b013e3181ace703
Croskerry, P., & Nimmo, G. R. (2011). Better clinical decision making and reducing diagnostic
error. Journal of the Royal College of Physicians of Edinburgh, 41(2), 155–162.
doi:10.4997/JRCPE.2011.208
Croskerry, P., & Norman, G. (2008). Overconfidence in clinical decision making. American
Journal of Medicine, 121(5 SUPPL.), 24–29. doi:10.1016/j.amjmed.2008.02.001
Cutrer, W. B., Sullivan, W. M., & Fleming, A. E. (2013). Educational strategies for improving
147
clinical reasoning. Current Problems in Pediatric and Adolescent Health Care, 43(9), 248–
257. doi:10.1016/j.cppeds.2013.07.005
Davis, E. A. (2003). Prompting middle school science students for productive reflection: Generic
and directed prompts. Journal of the Learning Sciences, 12(1), 91–142.
doi:10.1207/S15327809JLS1201_4
Davis, E. A., & Linn, M. C. (2000). Scaffolding students’ knowledge integration: prompts for
reflection in KIE. International Journal of Science Education, 22(8), 819–837.
doi:10.1080/095006900412293
Eddy, D. M. (1990). Clinical decision making: from theory to practice. JAMA : The Journal of
the American Medical Association, 263(13), 1839–1841. doi:10.1001/jama.263.13.1839
Elstein, A. S., Schwartz, A., & Schwarz, A. (2002). Clinical problem solving and diagnostic
decision making: selective review of the cognitive literature. British Medical Journal, 324,
729–732. doi:10.1136/bmj.324.7339.729
Eng, J. (2003). Sample size estimation: how many individuals should be studied? Radiology,
227(2), 309–313. doi:10.1148/radiol.2272012051
Epstein, R. M. (1999). Mindful practice. JAMA : The Journal of the American Medical
Association, 282(9), 833–9. doi:10.1001/jama.282.9.833
Ertmer, P. A., & Russell, J. D. (1995). Using case studies to enhance instructional design
education. Educational Technology, 35(4), 23–31.
Evans, J. S. B. T. (2003). In two minds: Dual-process accounts of reasoning. Trends in Cognitive
Sciences, 7(10), 454–459. doi:10.1016/j.tics.2003.08.012
Evans, J. S. B. T., & Over, D. E. (1996). Rationality and reasoning. Hove, UK: Psychology
Press.
148
Field, A. (2013). Discovering statistics using IBM SPSS statistics. (M. Carmichael, Ed.) (Kindle
Edi). SAGE Publications.
Fletcher, O. J., Hooper, B. E., & Schoenfeld-Tacher, R. (2015). Instruction and curriculum in
veterinary medical education: A 50-year perspective. Journal of Veterinary Medical
Education, 42(5), 489–500. doi:10.3138/jvme.0515-071
Flynn, A. E., & Klein, J. D. (2001). The influence of discussion groups in a case-based learning
environment. Educational Technology Research and Development, 49(3), 71–86.
Ge, X., & Land, S. M. (2003). Scaffolding students’ problem-solving processes in an ill-
structured task using question prompts and peer interactions. Educational Technology
Research and Development, 51(1), 21–38. doi:10.1007/BF02504515
Gee, J. P. (1997). Thinking, learning, and reading: The situated sociocultural mind. In D.
Kirshner & J. A. Whitson (Eds.), Situated cognition: Social, semiotic, and psychological
perspectives (pp. 235–260). Mahwah, NJ: Lawrence Erlbaum Associates.
Geweke, J. F., & Singleton, K. J. (1980). Interpreting the likelihood ratio statistic in factor
models when sample size is small. Journal of the American Statistical Association, 75(369),
133–137. doi:10.2307/2287400
Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the
effectiveness of peer feedback for learning. Learning and Instruction, 20(4), 304–315.
doi:10.1016/j.learninstruc.2009.08.007
Gielen, S., Tops, L., Dochy, F., Onghena, P., & Smeets, S. (2010). A comparative study of peer
and teacher feedback and of various peer feedback forms in a secondary school writing
curriculum. British Educational Research Journal, 36(1), 143–162.
doi:10.1080/01411920902894070
149
Hammond, K. R. (1996). Copng with uncertainty: The rivalry between intuition and analysis. In
K. R. Hammond (Ed.), Human judgment and social policy: Inrreducible uncertainty,
inevitable error, unavoidable injustice (pp. 60–93). New York: Oxford University Press.
Hansen, W. F., Ferguson, K. J., Sipe, C. S., & Sorosky, J. (2005). Attitudes of faculty and
students toward case-based learning in the third-year obstetrics and gynecology clerkship.
American Journal of Obstetrics and Gynecology, 192(2), 644–647.
doi:10.1016/j.ajog.2004.10.595
Harasym, P. M. H. J. A. W. W. (1997). Helping students to learn to think like experts when
solving clinical problems. Academic Medicine, 72(3), 173–179.
Harbison, J. (1991). Clinical decision making in nursing. Journal of Advanced Nursing, 16(4),
404–407. doi:10.1111/j.1365-2648.1991.tb03429.x
Hardin, L. E. (2003a). Problem-solving concepts and theories. Journal of Veterinary Medical
Education, 30(3), 226–229.
Hardin, L. E. (2003b). Research in medical problem solving: a review. Journal of Veterinary
Medical Education, 30(3), 230–235. doi:10.3138/jvme.30.3.230
Higgs, J., & Jones, M. A. (2008). Clinical decision making and multiple problem spaces. In J.
Higgs, M. Jones, S. Loftus, & Nicole Christensen (Eds.), Clinical reasoning in the health
professions (3rd Editio, p. Chapter 1). Retrieved from amazon.com
Jenkins, H. M. (1985). Improving clinical decision making in nursing. The Journal of Nursing
Education, 24(6), 242–243.
Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured
problem-solving learning outcomes. Educational Technology Research and Development,
45(1), 65–94.
150
Jonassen, D. H. (2010). Research Issues in Problem Solving. In The 11th International
Conference on Education Research: New Educational Paradigm for Learning and
Instruction (pp. 1–15).
Jonassen, D. H., & Hernandez-Serrano, J. (2002). Case-based reasoning and instructional design:
Using stories to support problem solving. Educational Technology Research and
Development, 50(2), 65–77. doi:10.1007/BF02504994
Jonassen, D. H., & Hung, W. (2008). All problems are not equal: Implications for problem-based
learning. Interdisciplinary Journal of Problem-Based Learning, 2(2), 10–13.
doi:10.7771/1541-5015.1080
Jones, M. a. (1992). Clinical reasoning in manual therapy. Physical Therapy, 72(12), 875–884.
doi:1454863
Kahneman, D., & Frederick, S. (2005). A model of heuristic judgment. In K. J. Holyoak &
Robert G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 267–
294). New York, NY: Cambridge University Press.
Kassirer, J. P. (2010). Teaching clinical reasoning: case-based and coached. Academic
Medicine : Journal of the Association of American Medical Colleges, 85(7), 1118–1124.
doi:10.1097/ACM.0b013e3181d5dd0d
Khosa, D. K., Volet, S. E., & Bolton, J. R. (2014). Making clinical case-based learning in
veterinary medicine visible: Analysis of collaborative concept-mapping processes and
reflections. Journal of Veterinary Medical Education, 41(4), 406–417.
doi:10.3138/jvme.0314-035R1
King, L., & MacLeod, M. (2002). The role of intuition and the development of expertise in
surgical ward and intensive care nurses. Journal of Advanced Nursing, 37, 322–329.
151
Kitchener, K. S. (1983). Cognition, metacognition, and epistemic cognition: a three-level model
of cognitive processing. Human Development, 4, 222–232.
Klein, G. A. (1993). A recognition-primed decision (RPD) model of rapid decision making. In G.
A. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action:
Models and methods (pp. 138–147). Norwood, NJ: Ablex Publishing Corporation.
Klein, G. A. (2008). Naturalistic decision making. Human Factors, 50(3), 456–460.
doi:10.1518/001872008X288385.
Klein, G. A., & Klinger, D. (1991). Naturalistic decision making. Human Systems IAC Gateway,
2(1), 16–19.
Kolodner, J. L., Hmelo, C. E., & Narayanan, N. H. (1996). Problem-Based Learning Meets Case-
Based Reasoning. In ICLS ’96 Proceedings of the 1996 international conference on
Learning sciences (pp. 188–195).
Kolodner, J. L., Owensby, J. N., & Guzdial, M. (2004). Case-based learning aids. In D. H.
Jonassen (Ed.), Handbook of Research for Education Communications and Technology
(2nd Editio, Vol. 2, pp. 829–861). Mahwah, NJ: Lawrence Erlbaum Associates. Retrieved
from http://www.aect.org/edtech/32.pdf
Ladyshewsky, R., & Jones, M. A. (2008). Peer coaching to generate clinical reasoning skills. In
J. Higgs, M. A. Jones, S. Loftus, & N. Christensen (Eds.), Clinical reasoning in the health
professions (3rd Editio). Elsevier Health Sciences. Retrieved from amazon.com
Land, S. M., & Zembal-Saul, C. (2003). Scaffolding reflection and articulation of scientific
explanations in a data-rich, project-based learning environment: An investigation of
progress portfolio. Educational Technology Research and Development, 51(4), 65–84.
doi:10.1007/BF02504544
152
LeBoeuf, R. A., & Shafir, E. B. (2005). Decision making. In K. J. Holyoak & R. G. Morrison
(Eds.), The Cambridge handbook of thinking and reasoning (pp. 243–265). Cambridge
University Press.
Lilliefors, H. W. (1967). On the Kolmogorov-Smirnov test for normality with mean and variance
unknown. Journal of the American Statistical Association, 62(318), 399–402.
doi:10.1080/01621459.1967.10482916
Linn, M. (2000). Designing the knowledge integration environment. International Journal of
Science Education, 22(8), 781–796.
Lipshitz, R. (1993). Converging themes in the study of decision making in realistic settings. In
G. A. Klein, J. Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in
action: Models and methods (pp. 103–137). Norwood, NJ: Ablex Publishing Corporation.
Lipshitz, R., Klein, G. A., Orasanu, J., & Salas, E. (2001). Taking stock of naturalistic decision
making. Journal of Behavioral Decision Making, 14(5), 331–352. doi:10.1002/bdm.381
Machiko R. Tomita. (2006). Methods of analysis: from univariate to multivariate statistics. In G.
Kielhofner (Ed.), Research in occupational therapy: Methods of inquiry for enhancing
practice (pp. 243–280). PA: Philadelphia: F.A. Davis Company.
Mamede, S., & Schmidt, H. G. (2004). The structure of reflective in medicine. Medical
Education, 38(12), 1302–1308. doi:10.1111/j.1365-2929.2004.01917.x
Mamede, S., & Schmidt, H. G. (2005). Correlates of reflective practice in medicine. Advances in
Health Sciences Education, 10(4), 327–337. doi:10.1007/s10459-005-5066-2
Mann, K., Gordon, J., & MacLeod, A. (2009). Reflection and reflective practice in health
professions education: A systematic review. Advances in Health Sciences Education, 14(4),
595–621. doi:10.1007/s10459-007-9090-2
153
Maudsley, G., & Strivens, J. (2000). Promoting professional knowledge, experiential learning
and critical thinking for medical students. Medical Education, 34(7), 535–544.
doi:10.1046/j.1365-2923.2000.00632.x
May, S. (2013). Clinical reasoning and case-based decision making: the fundamental challenge
to veterinary educators. Journal of Veterinary Medical Education, 40(3), 200–209.
doi:10.3138/jvme.0113-008R
McKenzie, B. (2014). Veterinary clinical decision-making: cognitive biases, external constraints,
and strategies for improvement. Journal of the American Veterinary Medical Association,
244(3), 271–276. doi:10.2460/javma.244.3.271
Moon, J. A. (2004). A handbook of reflective and experiential learning: Theory and practice.
Oxon, OX: RoutledgeFalmer.
Mory, E. H. (2004). Feedback research revisited. In D. H. Jonassen (Ed.), Handbook of research
on educational communications and technology (2nd Editio, pp. 745–783). Mahwah, NJ:
Lawrence Erlbaum Associates.
Ntoumanis, N., & Myers, N. D. (2016). An introduction to intermediate and advanced statistical
analyses for sport and exercise scientists. John Wiley & Sons.
Orasanu, J., & Connolly, T. (1993). The reinvention of decision making. In G. A. Klein, J.
Orasanu, R. Calderwood, & C. E. Zsambok (Eds.), Decision making in action: Models and
methods (pp. 3–20). Norwood, NJ: Alex Publishing Corporation.
Patel, V. L., Arocha, J. F., & Zhang, J. (2005). Thinking and reasoning in medicine. In K. J.
Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp.
281–286). Cambridge: Cambridge University Press.
Patton, D. D. (1978). Introduction to clinical decision making. Seminars in Nuclear Medicine,
154
8(4), 273–282.
Pedersen, S., & Liu, M. (2002). The effects of modeling expert cognitive strategies during
problem-based learning. Journal of Educational Computing Research, 26(4), 353–380.
doi:10.2190/6NL3-HMED-J8HE-GD4T
Quintana, C., Zhang, M., & Krajcik, J. (2005). A framework for supporting metacognitive
aspects of online inquiry through software-based scaffolding. Educational Psychologist,
40(4), 235–244. doi:10.1207/s15326985ep4004_5
Rashotte, J., & Carnevale, F. a. (2004). Medical and nursing clinical decision making: a
comparative epistemological analysis. Nursing Philosophy : An International Journal for
Healthcare Professionals, 5(2), 160–174. doi:10.1111/j.1466-769X.2004.00175.x
Riegger, M. H. (2011, June). Using S.O.A.P. is good medicine. DVM360 Magazine, 1–2.
Retrieved from http://veterinarynews.dvm360.com/print/327899?page=full
Rogoff, B. (1990). Peer interaction and cognitive development. In Apprenticeship in thinking:
Cognitive development in social context (pp. 171–188). New York, NY: Oxford University
Press.
Sato, M. (2013). Beliefs about peer interaction and peer corrective feedback: Efficacy of
classroom intervention. Modern Language Journal, 97(3), 611–633. doi:10.1111/j.1540-
4781.2013.12035.x
Schon, D. A. (1983). The reflective practitioner: How professionals think in action. New York:
Basic Books.
Schon, D. A. (1988). From technical rationality to reflection-in-action. In J. A. Dowie & A. S.
Elstein (Eds.), Professional judgment: A reader in clinical decision making (pp. 60–77).
New York, NY: Cambridge University Press. doi:10.1097/00006247-198308000-00009
155
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental
designs for generalized causal inference. Belmont, CA: Wadsworth.
Shin, N., Jonassen, D. H., & McGee, S. (2003). Predictors of well-structured and ill-structured
problem solving in an astronomy simulation. Journal of Research in Science Teaching,
40(1), 6–33. doi:10.1002/tea.10058
Slovic, P., Finucane, M. L., Peters, E., & MacGregor, D. G. (2004). Risk as analysis and risk as
feelings. Risk Analysis, 24(2). Retrieved from papers2://publication/uuid/F4B3EA59-C2F7-
4F62-9223-701C27F152C2
Smith, M., Higgs, J., & Elizabeth Ellis. (2008). Factors influencing clinical decision making. In
J. Higgs, M. Jones, S. Loftus, & N. Christensen (Eds.), Clinical reasoning in the health
professions (3rd Editio). Retrieved from amazon.com
Song, H.-D., Grabowski, B. L., Koszalka, T. a., & Harkness, W. L. (2006). Patterns of
instructional-design factors prompting reflective thinking in middle-school and college level
problem-based learning environments. Instructional Science, 34(1), 63–87.
doi:10.1007/s11251-005-6922-4
Spiro, R. J., Coulson, R. L., Feltovich, P. J., & Anderson, D. K. (1988). Cognitive flexibility
theory: Advanced knowledge acquisition in ill-structured domains. Champain, Illinois.
Swanson, R. A., & Holton, E. F. (2005). Research in Organizations: Foundations and Methods
in Inquiry. San Francisco: Berrett-Koehler Publishers.
Terry, W., & Higgs, J. (1993). Educational programmes to develop clinical reasoning skills.
Australian Journal of Physiotherapy, 39(1), 47–51. doi:10.1016/S0004-9514(14)60469-4
Thistlethwaite, J. E., Davies, D., Ekeocha, S., Kidd, J. M., MacDougall, C., Matthews, P., …
Clay, D. (2012). The effectiveness of case-based learning in health professional education.
156
Medical Teacher, 34(6), 421–444. doi:10.3109/0142159X.2012.680939
Thomas, S. A., Wearing, A. J., & Bennett, M. J. (1991). Clinical decision making for nurses and
health professionals. Sydney: W B Saunders/Ballière Tindall.
Thompson, C., & Dowding, M. (2002). Clinical Decision Making and Judgement in Nursing.
London: Churchill Livingstone.
Thurman, J., Volet, S. E., & Bolton, J. R. (2009). Collaborative, Case-based Learning: How Do
Students Actually Learn from Each Other? Journal of Veterinary Medical Education, 36(3),
297–304. doi:10.3138/jvme.36.3.297
Uribe, D., Klein, J. D., & Sullivan, H. (2003). The effect of computer-mediated collaborative
learning on solving III-defined problems. Educational Technology Research and
Development, 51(1), 5–19. doi:10.1007/BF02504514
van den Boom, G., Paas, F. G. W. C., & van Merriënboer, J. J. G. (2007). Effects of elicited
reflections combined with tutor or peer feedback on self-regulated learning and learning
outcomes. Learning and Instruction, 17(5), 532–548.
doi:10.1016/j.learninstruc.2007.09.003
van der Pol, J., van den Berg, B. a M., Admiraal, W. F., & Simons, P. R. J. (2008). The nature,
reception, and use of online peer feedback in higher education. Computers and Education,
51(4), 1804–1817. doi:10.1016/j.compedu.2008.06.001
Van Manen, M. (1991). Reflectivity and the pedagogical moment: the normativity of
pedagogical thinking and acting. Journal of Curriculum Studies, 23, 507–536.
Vandeweerd, J.-M., Vandeweerd, S., Gustin, C., Keesemaecker, G., Cambier, C., Clegg, P., …
Gustin, P. (2012a). Clinical reasoning and decision analysis. Journal of Veterinary Medical
Education, 39(2), 142–151. doi:10.3138/jvme.0911.098R1
157
Vandeweerd, J.-M., Vandeweerd, S., Gustin, C., Keesemaecker, G., Cambier, C., Clegg, P., …
Gustin, P. (2012b). Understanding Veterinary Practitioners’ Decision-Making Process:
Implications for Veterinary Medical Education. Journal of Veterinary Medical Education,
39(2), 142–151. doi:10.3138/jvme.0911.098R1
Volet, S., Summers, M., & Thurman, J. (2009). High-level co-regulation in collaborative
learning: How does it emerge and how is it sustained? Learning and Instruction, 19(2),
128–143. doi:10.1016/j.learninstruc.2008.03.001
Whitney, M., Herron, M., & Weeks, B. (1993). Preclinical curricular alternatives: history and
rationale of problem- based medical education. J Vet Med Educ, 20, 2–8.
Williams, S. M. (1992). Putting case-based instruction into context: Examples from legal and
medical education. Journal of the Learning Sciences, 2(4), 367–427.
doi:10.1207/s15327809jls0204_2
Wojcikowski, K., & Brownie, S. (2013). Generic reflective feedback : An effective approach to
developing clinical reasoning skills. Journal of Computer Assisted Learning, 29, 371–382.
doi:10.1111/jcal.12012
Wood, D., Bruner, J. S., & Ross, G. (1976). The role of tutoring in problem solving. Journal of
Child Psychology and Child Psychiatry, 17, 89–100. Retrieved from
http://onlinelibrary.wiley.com/doi/10.1111/j.1469-7610.1976.tb00381.x/abstract
Yang, Y.-F. (2011). A reciprocal peer review system to support college students’ writing. British
Journal of Educational Technology, 42(4), 687–700. doi:10.1111/j.1467-
8535.2010.01059.x
Yates, J., Veinott, E., & Patalano, A. (2003). Hard decisions, bad decisions: On decision quality
and decision aiding. In S. L. Schneider & J. Shanteau (Eds.), Emerging Perspectives in
158
Judgment and Decision Research (pp. 13–63). New York: Cambridge University Press.
Retrieved from http://works.bepress.com/andrea_patalano/18/
159
APPENDIX A
PEER FEEDBACK GUIDELINES WITH REFLECTIVE PROMPTS
160
161
162
163
APPENDIX B
ONLINE SURVEY
164
top related