![Page 1: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/1.jpg)
Dylan Wiliam (@dylanwiliam)
Formative Assessment: The most effective way to build learning
www.dylanwiliamcenter.com
![Page 2: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/2.jpg)
Outline: four questions
• Why we need to raise achievement
• Why we have to focus our efforts
• Why formative assessment should be the priority
• Why formative assessment requires new models of professional development
2
![Page 3: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/3.jpg)
Why we need to raise achievement3
• In advanced economies, over the next 20 to 30 years
– Between a quarter and a third of jobs could be offshored (Blinder, 2011)
– About half of all jobs could be done by machines (Frey & Osborne, 2013)
• The choice for young people
– Wait for someone else to invent a new job for you
– Create your own
![Page 4: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/4.jpg)
Change 1969-1999
+14%
+8%
–5%
–8%
–3%
The world of work is changing
Skill category
A Complex communication
B Expert thinking/problem solving
C Non-routine manual
D Routine cognitive
E Routine manual
Which kinds of skill are disappearing fastest from the workplace?
4
Autor, Levy and Murnane (2003)
![Page 5: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/5.jpg)
What kinds of schools do we need?
School model Ethos Key process
Talent refineries School must provide opportunities for students to show what they can do
Ensuring good teaching and syllabus coverage
Talent incubators All students students can learn, but not all students can achieve at high levels
Drawing out what is within the student
Talent factories All students can achieve at high levels
“Whatever it takes”
![Page 6: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/6.jpg)
Why every school should do pareto analysis6
• Vilfredo Pareto (1848-1923)
– Economist, philosopher, and sociologist,
associated with the 80:20 rule
• Pareto improvement
– A change that can make at least one person
(e.g., a student) better off without making
anyone else (e.g., a teacher) worse off.
• Pareto efficiency/Pareto optimality
– An allocation (e.g., of resources) is Pareto efficient or Pareto
optimal when there are no more Pareto improvements
![Page 7: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/7.jpg)
Using research evidence
7
![Page 8: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/8.jpg)
Relevant studies
• Fuchs & Fuchs (1986)
• Natriello (1987)
• Crooks (1988)
• Bangert-Drowns et al. (1991)
• Dempster (1991)
• Dempster (1992)
• Elshout-Mohr (1994)
• Kluger & DeNisi (1996)
• Black & Wiliam (1998)
• Nyquist (2003)
• Brookhart (2004)
• Allal & Lopez (2005)
• Köller (2005)
• Brookhart (2007)
• Wiliam (2007)
• Hattie & Timperley (2007)
• Shute (2008)
• Kingston & Nash (2011, 2015)
8
![Page 9: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/9.jpg)
Formative Assessment: A contested term
Span
Length
Impact
Long-cycle Medium-cycle Short-cycle
Across terms, teaching units
Four weeks toone year
Monitoring, curriculum alignment
Within and between lessons
Minute-by-minute and day-by-day
Engagement, responsiveness
Within and between
teaching units
One to four weeks
Student-involved
assessment
9
![Page 10: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/10.jpg)
Where the learner is going
Where the learneris now
How to get the learner there
Teacher
Peer
Student
Unpacking Formative Assessment
Clarifying, sharing, and
understanding learning
intentions
Eliciting evidence of learning
Providing feedback that
moves learners forward
Activating students as learningresources for one another
Activating students asowners of their own learning
10
![Page 11: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/11.jpg)
Where the learner is going
Where the learneris now
How to get the learner there
Teacher
Peer
Student
Unpacking Formative Assessment11
Using evidence of achievement to adapt what
happens in classrooms to meet learner needs
![Page 12: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/12.jpg)
An educational positioning system
• A good teacher:
– Establishes where the students are in their learning
– Identifies the learning destination
– Carefully plans a route
– Begins the learning journey
– Makes regular checks on progress on the way
– Makes adjustments to the course as conditions dictate
12
![Page 13: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/13.jpg)
Educational Endowment Foundation toolkit13
Intervention Cost Quality of evidence
Extra months of learning
Feedback ££ +8
Metacognition and self-regulation ££ +8
Peer tutoring ££ +6
Early years intervention £££££ +6
One to one tuition ££££ +5
Homework (secondary) £ +5
Collaborative learning £ +5
Phonics £ +4
Small group tuition £££ +4
Behaviour interventions £££ +4
Digital technology ££££ +4
Social and emotional learning £ +4
![Page 14: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/14.jpg)
Educational Endowment Foundation toolkit14
Intervention Cost Quality of evidence
Extra months of learning
Parental involvement £££ +3
Reducing class size £££££ +3
Summer schools £££ +3
Sports participation £££ +2
Arts participation ££ +2
Extended school time £££ +2
Individualized instruction £ +2
After school programmes ££££ +2
Learning styles £ +2
Mentoring £££ +1
Homework (primary) £ +1
![Page 15: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/15.jpg)
Educational Endowment Foundation toolkit15
Intervention Cost Quality of evidence
Extra months of learning
Teaching assistants ££££ 0
Performance pay ££ 0
Aspiration interventions £££ 0
Block scheduling £ 0
School uniform £ 0
Physical environment ££ 0
Ability grouping £ -1
Educational Endowment Foundation (2015)
![Page 16: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/16.jpg)
Where the learner is going
Where the learneris now
How to get the learner there
Teacher
Peer
Student
Unpacking Formative Assessment
Clarifying, sharing, and
understanding learning
intentions
Engineering effective discussions, tasks, and activities that elicit evidence of
learning
Providing feedbackthat moves learners
forward
Activating students asresources for one another
Activating students asowners of their own learning
16
![Page 17: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/17.jpg)
Strategies and practical techniques for classroom formative assessment
17
![Page 18: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/18.jpg)
Clarifying, sharing and understanding learning intentions
18
![Page 19: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/19.jpg)
Share learning intentions
• Explain learning intentions at start of lesson/unit:– Learning intentions
– Success criteria
• Consider providing learning intentions and success criteria in students’ language
• Use posters of key words to talk about learning:– E.g., describe, explain, evaluate
• Use planning and writing frames judiciously
• Use annotated examples of different standards to “flesh out” assessment rubrics (e.g., lab reports).
• Provide opportunities for students to design their own tests.
19
![Page 20: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/20.jpg)
Engineering effective discussions, activities, and classroom tasks that elicit evidence of learning
20
![Page 21: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/21.jpg)
Eliciting evidence
• Key idea: questioning should
– cause thinking
– provide data that informs teaching
• Improving teacher questioning
– generating questions with colleagues
– low-order vs. high-order not closed vs. open
– appropriate wait-time
• Getting away from I-R-E (initiation-response-evaluation)
– basketball rather than serial table-tennis
– ‘No hands up’ (except to ask a question)
– ‘Hot Seat’ questioning
• All-student response systems
– ABCD cards, “show-me” boards, exit passes
21
![Page 22: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/22.jpg)
Providing feedback that moves learners forward
22
![Page 23: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/23.jpg)
Effects of feedback
• Kluger & DeNisi (1996) review of 3000 research reports
• Excluding those:
– without adequate controls
– with poor design
– with fewer than 10 participants
– where performance was not measured
– without details of effect sizes
• left 131 reports, 607 effect sizes, involving 12652 individuals
• On average, feedback increases achievement
– Effect sizes highly variable
– 38% (231 out of 607) of effect sizes were negative
23
![Page 24: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/24.jpg)
Getting feedback right is hard
Response type Feedback indicates performance…
falls short of goal exceeds goal
Change behavior Increase effort Exert less effort
Change goal Reduce aspiration Increase aspiration
Abandon goal Decide goal is too hard Decide goal is too easy
Reject feedback Feedback is ignored Feedback is ignored
24
![Page 25: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/25.jpg)
Provide feedback that moves learning on
• Key idea: feedback should:
– Cause thinking
– Provide guidance on how to improve
• Comment-only marking
• Focused marking
• Explicit reference to mark-schemes/rubrics
• Suggestions on how to improve:
– Not giving complete solutions
• Re-timing assessment:
– E.g., three-fourths-of-the-way-through-a-unit test
25
![Page 26: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/26.jpg)
Activating students as learning resources for one another
26
![Page 27: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/27.jpg)
Help students be learning resources
• Students assessing their peers’ work:
– “Pre-flight checklist”
– “Two stars and a wish”
– Choose-swap-choose
– Daily sign-in
• Training students to pose questions/identifying group weaknesses
• End-of-lesson students’ review
27
![Page 28: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/28.jpg)
Activating students as owners of their own learning
28
![Page 29: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/29.jpg)
Help students own their own learning
• Students assessing their own work:
– With rubrics
– With exemplars
• Self-assessment of understanding:
– Learning portfolio
– Traffic lights
– Red/green discs
– Coloured cups
– Plus/minus/interesting
29
![Page 30: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/30.jpg)
So much for the easy bit
![Page 31: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/31.jpg)
A model for teacher learning
• Content, then process
• Content (what we want teachers to change):
– Evidence
– Ideas (strategies and techniques)
• Process (how to go about change):
– Choice
– Flexibility
– Small steps
– Accountability
– Support
31
![Page 32: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/32.jpg)
Supportive accountability
• What is needed from teachers:
– A commitment to:• The continual improvement of practice
• Focus on those things that make a difference to students
• What is needed from leaders:
– A commitment to engineer effective learning environments for teachers by:• Creating expectations for continually improving practice
• Keeping the focus on the things that make a difference to students
• Providing the time, space, dispensation, and support for innovation
• Supporting risk-taking
32
![Page 33: Formative Assessment: The most effective way to build learning](https://reader031.vdocument.in/reader031/viewer/2022022519/6217ce7178d27b6462286d27/html5/thumbnails/33.jpg)
To find out more…
www.dylanwiliam.net
www.dylanwiliamcenter.com
33