principles of feedback delivery and the role of self- assessment
DESCRIPTION
Principles of feedback delivery and the role of self- assessment. Kevin W. Eva Department of Clinical Epidemiology and Biostatistics Program for Educational Research and Development McMaster University. Principles of feedback delivery. There are no universal principles. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/1.jpg)
Principles of feedback delivery and the
role of self-assessment
Kevin W. EvaDepartment of Clinical
Epidemiology and Biostatistics
Program for Educational Research and Development
McMaster University
![Page 2: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/2.jpg)
Principles of feedback delivery
1. There are no universal principles
![Page 3: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/3.jpg)
“There is no “best” type of formative feedback for all learners and all
learning outcomes”Shute (2008)
Click icon to add picture
![Page 4: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/4.jpg)
Principles of feedback delivery
1. There are no universal principles
2. Self-assessment is insufficient
![Page 5: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/5.jpg)
The Rhetoric
• Almost every article on self-assessment begins with the same basic sentence:
“The ability to self-assess is a cornerstone of professional self-regulation”
![Page 6: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/6.jpg)
The self-regulating professional
•Reflect regularly on daily practice
•Self-assess gaps in knowledge or skill
•Seek opportunities to redress gaps
•Invest energy to learn (or relearn)
•Incorporate knowledge into practice
•Repeat(Handfield-Jones, et al., 2002)
![Page 7: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/7.jpg)
Eva, et al. (2004)
Before After Previous-0.2
0
0.2
0.4
0.6
0.8
1
Year 1Year 2Year 3
Corr
elat
ion
(r)
![Page 8: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/8.jpg)
The self-assessment literature
• Hundreds of articles• Many literature reviews• One conclusion
Self-assessment ability
is generally poor
![Page 9: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/9.jpg)
Boehler, Rogers, et al. (2006)
Pre-test Post-instruction Post-feedback0
5
10
15
20
25
Compliment group Feedback group
Perf
orm
ance
Rat
ing
![Page 10: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/10.jpg)
“In a majority of relevant studies
physicians do not appear to
accurately self-assess”
Davis, et al. (2006)
![Page 11: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/11.jpg)
The self-assessment literature
•Either the conclusions of hundreds of studies using multiple methodologies are all wrong or a critical premise underlying the concept of self-regulation in the professions is unsupportable
![Page 12: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/12.jpg)
If you don’t think you’re susceptible, you’re not alone …
FAE Halo effect Cognitive dissonance
Reactive devaluation
12345678
Average Population Self Rating
Pronin, Lin, Ross (2002)
…
![Page 13: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/13.jpg)
“Evolution deserves the Microsoft Windows Award for installing these mental processes in every one of us without asking permission.”
(Gilbert, 2006)
![Page 14: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/14.jpg)
Principles of feedback delivery
1. There are no universal principles
2. Self-assessment is insufficient
3. Self-assessment is vital
![Page 15: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/15.jpg)
“It is impossible for a man to learn
what he thinks he already knows”
Epictetus (circa 100AD)
Click icon to add picture
![Page 16: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/16.jpg)
Quality of
feed-back
Eval of student
Self-evalua-
tion
Station order
Station The Student
00.050.1
0.150.2
0.250.3
0.35
Stan
dard
ized
beta
Predicting generation of learning goals
(Overall R2=.45; Eva, et al., In press)
![Page 17: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/17.jpg)
“Physician agreement with
colleagues’ scores correlated
positively with overall mean scores from colleagues (R=.48)”Sargeant et al.
(2003)
Click icon to add picture
![Page 18: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/18.jpg)
The ironic nature of self-assessment
• We can’t trust it … but we should encourage it
• We need to influence it … without changing it
![Page 19: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/19.jpg)
Self Efficacy• Influenced by feedback, but also influences
future performance
• Not merely a passive reflection of performance, but part of a self-fulfilling prophecy that affects performance
(see Dweck, 1999)
![Page 20: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/20.jpg)
Implications for self-regulation
• “Rose colored glasses” approach to reflection understandable and necessary• Important for ability to function and succeed
• Can never create a “good self-assessor” • And probably shouldn’t try
![Page 21: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/21.jpg)
Principles of feedback delivery
1. There are no universal principles
2. Self-assessment is insufficient3. Self-assessment is vital4. The goal is to enable
discovery
![Page 22: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/22.jpg)
Age Diagnosis/ health risk
Lab results Medication use
0102030405060708090
Type of patient information
% o
f phy
sici
ans
% of physicians for whom it is difficult/impossible to generate
practice data about…
(Audet, et al., 2005)
![Page 23: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/23.jpg)
One 2 to 9 10 to 49 50 or more05
101520253035404550
Number of physicians per practice
% o
f phy
sici
ans
% of physicians who have access to any quality of care data
(Audet, et al., 2005)
![Page 24: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/24.jpg)
The need for difficulties• Learners tend to request feedback after
successful experiences(Chiviacowsky and Wulf, 2002)
• Learning strategies perceived as beneficial are often sub-optimal
(Simon and Bjork, 2001)• Proficiency tends to be inferred from the
rate of learning rather than performance(Eva, 2009)
![Page 25: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/25.jpg)
Jowett, et al. (2007)
Learner
Material
![Page 26: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/26.jpg)
Test-enhanced learning
5 Minutes 2 Days 1 Week0
0.10.20.30.40.50.60.70.80.9
Study, StudyStudy, Test
Retention Interval
Perc
ent R
ecal
l
See Larsen, Butler, and Roediger (2008)
![Page 27: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/27.jpg)
“The phrase self-assessment should not imply an isolated or individualistic activity; it should commonly involve peers, teachers, and other sources of information”
(Boud, 1999)
![Page 28: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/28.jpg)
“Education is learning what you didn’t even know you didn’t know”
(Boorstin, 1914)
Click icon to add picture
![Page 29: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/29.jpg)
Principles of feedback delivery
1. There are no universal principles
2. Self-assessment is insufficient3. Self-assessment is vital4. The goal is to enable discovery5. Numbers can be harmful
![Page 30: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/30.jpg)
“Feedback interventions that draw attention to
the self via normative cues are largely ineffective”
(Kluger and DeNisi, 1998)
Click icon to add picture
![Page 31: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/31.jpg)
“Studies that augmented
feedback with consultation
produced substantially larger
differences, but other
methodological variations had little
effect”(Marsh and Roche,
1997)
Click icon to add picture
![Page 32: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/32.jpg)
Principles of feedback delivery
1. There are no universal principles2. Self-assessment is insufficient3. Self-assessment is vital4. The goal is to enable discovery5. Numbers can be harmful6. Credibility is likely to be the key
![Page 33: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/33.jpg)
Credibility
“With your clinical partner … they know your strengths and weaknesses and they’re probably going to give you quite useful feedback. Maybe a consultant watched you one time. They might not have the whole picture of what you’re capable of and what you normally do”
(Sargeant, et al., 2010)
![Page 34: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/34.jpg)
Credibility
Of relationships: Engagement and awareness
Of the process: Validity and authenticity
Of intent: Beneficence and non-maleficence
(Sargeant, et al., 2010)
![Page 35: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/35.jpg)
![Page 37: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/37.jpg)
Kruger and Dunning (1999)
![Page 38: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/38.jpg)
Kruger and Dunning (1999)
![Page 39: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/39.jpg)
• Self-assessment broadly conceived of as a prequel to self-directed learning activities aimed at performance improvements
• With respect to patient safety, we’re much more concerned about situation specific awareness (i.e., “reflection in practice”)• Continuous monitoring of whether or not one
has sufficient skills to effectively solve the current problem
The self-assessment literature(as summarized by Eva and Regehr, 2008)
![Page 40: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/40.jpg)
Self-monitoringKnowing when to defer• Participants clearly
knew when they were likely to respond in error
Round 1 Round 20
1020304050607080
Perc
ent
Corr
ect
(Eva and Regehr, 2007)
![Page 41: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/41.jpg)
Self-monitoring
(Eva and Regehr, 2007)
![Page 42: Principles of feedback delivery and the role of self- assessment](https://reader035.vdocument.in/reader035/viewer/2022062310/568163a8550346895dd4b911/html5/thumbnails/42.jpg)
Self-monitoring•Self-assessment (as overall judgment of ability) is importantly different from self-awareness (monitoring)
•Difficulty with self-assessment is not that we can’t judge our performance, but that we don’t aggregate well over events