the&secretof&effec:ve&feedback - dylanwiliamcenter.com ·...
TRANSCRIPT
Dylan Wiliam (@dylanwiliam)
The Secret of Effec:ve Feedback
www.dylanwiliamcenter.com www.dylanwiliam.net
Outline
• What is feedback? • Why most feedback research isn’t relevant • What we do know about feedback
• Prac:cal sugges:ons for improving feedback • Some thoughts about grading
2
What is feedback?
“ac:ons taken by (an) external agent (s) to provide informa:on regarding some aspect(s) of one’s task performance.” (Kluger & DeNisi, 1996 p. 255)
• Examples – Your typing speed is 45 words per minute
– You are 5 words per minute below proficiency – Use your thumb only to press the space bar
3
What is the purpose of feedback?
• Two main goals of feedback – To improve the work
– To improve the student
• Most of the :me, the purpose of feedback is to improve the work of students on tasks they have not yet a,empted.
4
Educa7onal Endowment Founda7on toolkit 5
Interven7on Cost Quality of evidence
Extra months of learning
Feedback $$ ★★★ +8
Metacogni:on and self-‐regula:on $$ ★★★★ +8
Peer tutoring $$ ★★★★ +6
Early years interven:on $$$$$ ★★★★ +6
One to one tui:on $$$$ ★★★★ +5
Homework (secondary) $ ★★★ +5
Collabora:ve learning $ ★★★★ +5
Phonics $ ★★★★ +4
Small group tui:on $$$ ★★★★ +4
Behaviour interven:ons $$$ ★★ +4
Digital technology $$$$ ★★★★ +4
Social and emo:onal learning $ ★★★★ +4
Educa7onal Endowment Founda7on toolkit 6
Interven7on Cost Quality of evidence
Extra months of learning
Parental involvement $$$ ★★★ +3
Reducing class size $$$$$ ★★★ +3
Summer schools $$$ ★★ +3
Sports par:cipa:on $$$ ★★ +2
Arts par:cipa:on $$ ★★★ +2
Extended school :me $$$ ★★ +2
Individualized instruc:on $ ★★★ +2
Aeer school programmes $$$$ ★★ +2
Learning styles $ ★★★ +2
Mentoring $$$ ★★★ +1
Homework (primary) $ ★★★ +1
Educa7onal Endowment Founda7on toolkit 7
Interven7on Cost Quality of evidence
Extra months of learning
Teaching assistants $$$$ ★★ 0
Performance pay $$ ★ 0
Aspira:on interven:ons $$$ ★ 0
Block scheduling $ ★★ 0
School uniform $ ★ 0
Physical environment $$ ★ 0
Ability grouping $ ★★★ -‐1
Kinds of feedback: Israel
• 264 low and high ability grade 6 students in 12 classes in 4 schools; analysis of 132 students at top and bojom of each class
• Same teaching, same aims, same teachers, same classwork • Three kinds of feedback: grades, comments, grades+comments
Butler (1988)
Achievement AEtude
Grades no gain High scorers: posi:ve
Low scorers: nega:ve Comments 30% gain High scorers: posi:ve
Low scorers: posi:ve
8
What happened for students given both grades and comments? A. Gain: 30%; Antude: all posi:ve B. Gain: 30%; Antude: high scorers posi:ve, low scorers nega:ve
C. Gain: 0%; Antude: all posi:ve D. Gain: 0%; Antude: high scorers posi:ve, low scorers nega:ve
E. Something else
Responses
Achievement AEtude
Grades no gain High scorers: posi:ve
Low scorers: nega:ve Comments 30% gain High scorers: posi:ve
Low scorers: posi:ve
9
Effects of feedback
• Kluger & DeNisi (1996) review of 3000 research reports • Excluding those:
– without adequate controls – with poor design – with fewer than 10 par:cipants – where performance was not measured
– without details of effect sizes • lee 131 reports, 607 effect sizes, involving 12652 individuals
• On average, feedback increases achievement – Effect sizes highly variable – 38% (231 out of 607) of effect sizes were nega:ve
10
Feedback in STEM subjects 11
• Review of 9000 papers on feedback in mathema:cs, science and technology
• Only 238 papers retained – Background papers 24
– Descrip:ve papers 79 – Qualita:ve papers 24
– Quan:ta:ve papers 111 • Mathema:cs 60
• Science 35
• Technology 16
Ruiz-‐Primo and Li (2013)
Classifica7on of feedback studies 12
1. Who provided the feedback (teacher, peer, self, or technology-‐based)? 2. How was the feedback delivered (individual, small group, or whole
class)? 3. What was the role of the student in the feedback (provider or
receiver)? 4. What was the focus of the feedback (e.g., product, process, self-‐
regula:on for cogni:ve feedback; or goal orienta:on, self-‐efficacy for affec:ve feedback)
5. On what was the feedback based (student product or process)? 6. What type of feedback was provided (evalua:ve, descrip:ve, or
holis:c)? 7. How was feedback provided or presented (wrijen, video, oral, or
video)? 8. What was the referent of feedback (self, others, or mastery criteria)? 9. How, and how oeen was feedback given in the study (one :me or
mul:ple :mes; with or without pedagogical use)?
Main findings 13
Characteris7c of studies included Maths Science
Feedback treatment is a single event las:ng minutes 85% 72%
Reliability of outcome measures 39% 63%
Validity of outcome measures 24% 3%
Dealing only or mainly with declara:ve knowledge 12% 36%
Schema:c knowledge (e.g., knowing why) 9% 0%
Mul:ple feedback events in a week 14% 17%
Kinds of feedback (Nyquist, 2003)
• Weaker feedback only – Knowledge or results (KoR)
• Feedback only – KoR + clear goals or knowledge of correct results (KCR)
• Weak forma:ve assessment – KCR+ explana:on (KCR+e)
• Moderate forma:ve assessment – (KCR+e) + specific ac:ons for gap reduc:on
• Strong forma:ve assessment – (KCR+e) + ac:vity
Effects of forma7ve assessment (HE)
Kind of feedback Count Effect
Weaker feedback only 31 0.14
Feedback only 48 0.36
Weaker forma:ve assessment 49 0.26
Moderate forma:ve assessment 41 0.39
Strong forma:ve assessment 16 0.56
GeEng feedback right is hard
Response type Feedback indicates performance…
falls short of goal exceeds goal
Change behavior Increase effort Exert less effort
Change goal Reduce aspira:on Increase aspira7on
Abandon goal Decide goal is too hard Decide goal is too easy
Reject feedback Feedback is ignored Feedback is ignored
16
Engagement and learning
• Ajribu:on (Dweck, 2000) – Personaliza:on (internal vs. external) – Permanence (stable vs. unstable) – Good learners ajribute failure and success to internal, unstable causes. (It’s down to you, and you can do something about it.)
• Views of ‘ability’ – Fixed (IQ) – Incremental (untapped poten:al)
– Essen:al that teachers inculcate in students a view that ‘ability’ is incremental rather than fixed (by working, you’re genng smarter).
Dual-‐pathway theory (Boekaerts, 2006)
• Long-‐term learning goals are translated into short-‐term learning inten:ons
• Dynamic comparisons of task and situa:onal demands with personal resources, taking into account: – Interest – Capability – Importance – Value – Cost
• Resul:ng ac:va:on of energy along one pathway: – Well-‐being – Growth
Summary of prac7cal techniques
• Focus on the reac:ons of the students, not the feedback – Build trust – “Wise feedback”
• Focus your students on growth, not preserving well-‐being
• Design feedback as part of a system • Make feedback into detec:ve work
19
Meanings and consequences of school grades
• Two ra:onales for grading – Meanings
• Assessment as eviden:ary reasoning • Assessment outcomes as supports for making inferences
– (e.g., about student achievement)
– Consequences • Assessment outcomes as rewards and punishments
• Assessments create incen:ves for students to do what we want them to do
– These two ra:onales interact, and conflict • achievement grades for comple:on of homework
• achievement grades for effort
• penal:es for late submission • zeroes for missing work
20
Effects of punishment 21
• 10 private day-‐care centers in Haifa, Israel • Each center opens at 7.30am and closes at 4.00pm • Each center normally takes up to 35 children
• A number of parents are late collec:ng children • 6 of the 10 centers started fining late parents – Fine for parents >10 minutes late: NIS10 per child
– For comparison, baby sijers earn NIS20 per hour
Gneezy and Rus:chini (2000)
“A fine is a price”
0
5
10
15
20
25
0 2 4 6 8 10 12 14 16 18 20
Num
ber o
f late-‐coming pa
rents
Week
Control New policy
Policy introduced in week 5 Policy abandoned in week 17
22
To find out more…
www.dylanwiliamcenter.com
23
To find out even more…
www.dylanwiliamcenter.com
24