g. gage kingsbury, dylan wiliam, & steven l. wise, nwea 11 th annual marces/msde conference...
TRANSCRIPT
![Page 1: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/1.jpg)
G. Gage Kingsbury, Dylan Wiliam, & G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEASteven L. Wise, NWEA
1111thth Annual MARCES/MSDE Conference Annual MARCES/MSDE Conference
University of Maryland, October, 2011University of Maryland, October, 2011
Connecting the Dots:Connecting the Dots:Formative, Interim, and Formative, Interim, and Summative AssessmentSummative Assessment
![Page 2: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/2.jpg)
Thomas Jefferson
“the ideal of offering all children the opportunity to succeed, regardless of who their parents happen to be” (Hirsch, 2006, p 30)
George Washington Carver
“Education is the key to unlock the golden door of freedom.”
Malcolm Forbes
“Education's purpose is to replace an empty mind with an open one.”
The purpose of education
![Page 3: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/3.jpg)
To provide each student with an opportunity to learn what life has available, to help them decide what interests them, and to help them learn as much as they can to take them in their desired direction.
Assumption: The mission of an educational system
![Page 4: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/4.jpg)
• Students – Am I doing well enough? – Can I reach my goals?
• Teachers – What is next for this student?– How am I doing?
• Parents – Is my child growing well?– How is my child doing relative to his/her peers?
• School Administrators – Is my school doing as well as it could? – Is my school meeting requirements?
Stakeholders and Assessment Needs
![Page 5: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/5.jpg)
• Legislators – Are we creating a strong workforce? – Will our next generation be ready to lead?
• The general public – Is my money being spent well? – Can I hire good employees?
Stakeholders and Assessment Needs (cont.)
![Page 6: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/6.jpg)
• Processes that allow a teacher to capture learning as it occurs (“formative”; ”diagnostic”)
• Procedures that allow the identification of student achievement and growth relative to the trait of primary interest during instruction (“interim”; “benchmark”).
• Procedures that allow identification of achievement at the end of a course of study (“summative”; “evaluative”).
Assessment tools for education
![Page 7: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/7.jpg)
• Currently, the primary focus of federal regulation is summative/evaluative assessment.
• This focus creates an imbalance in the classroom.
• We need to find balance, so each assessment tool can help us provide the educational outcomes we seek.
• Each approach provides only a portion of the information that stakeholders need
• Here we will describe the characteristics of each assessment tool, and how they might work together to provide more or the needed information
Using these tools together
![Page 8: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/8.jpg)
• Assesses student performance towards the end of a school year.
• Is primarily designed to make evaluative statements about groups of students.
• Measurement accuracy in the aggregate is crucial.
• Can be designed to make statements about individual students, but this requires longer tests.
The Annual Achievement Assessment (U.S. Version)
![Page 9: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/9.jpg)
• Neither detailed content coverage nor full-sample testing is required.
• For most purposes, immediacy of results is neither required nor available.
• Information shelf life (i.e., for how long can these data support the intended inferences?) is long.
• Needed frequency of data collection low
Characteristics
![Page 10: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/10.jpg)
• School administrators can use results to chart trends in student performance.
• Legislators can use results to identify funding needs.
• The general public can use results to gauge the effectiveness of the educational system.
• Teachers may be able to use results to help in curriculum planning for future cohorts.
Stakeholder Needs
![Page 11: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/11.jpg)
• Students: typically little to no actionable information about their instructional needs
• Teachers: little information about the instructional needs of this year’s individual students.
• Parents: little information about their student’s academic growth.
Unmet Needs
![Page 12: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/12.jpg)
• Focus on student achievement and growth relative to a trait of primary interest during instruction.
• Assesses student performance at multiple points during a year of instruction.
• Designed to support inferences about the academic growth of individual students.
• Measurement accuracy for the individual student is crucial. CAT is useful here.
Interim Assessment
![Page 13: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/13.jpg)
• All students are assessed.
• Immediacy of results is important.
• Can be interpreted normatively or relative to long-term goal (i.e., college readiness).
• Information shelf life is short.
• Frequency of data collection is moderate.
Characteristics
![Page 14: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/14.jpg)
• Students can gauge their growth relative to normative or aspirational goals.
• Parents can assess their child’s growth.• Teachers can use results to make instructional
decisions for the current cohort.• School administrators can aggregate student
results to assess trends in growth and to evaluate teacher effectiveness.
• Legislators can use results to evaluate policy.• The general public can use results to gauge the
effectiveness of the educational system.
Stakeholder Needs
![Page 15: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/15.jpg)
• Processes that allow a teacher to capture learning as it occurs, and to make appropriate instructional adjustments.
“Formative” Assessment
![Page 16: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/16.jpg)
• All students are assessed.
• Immediacy of results is important.
• Interpreted in terms of instructional decisions.
• Information shelf life is short.
• Frequency of data collection is high.
Characteristics
![Page 17: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/17.jpg)
Which of These is Formative?
A. District science supervisor uses test results to plan professional development workshops for teachers.
B. Teachers doing item-by-item analysis of 5th grade math tests to review their 5th grade curriculum.
C. A school tests students every 10 weeks to predict which students are “on course” to pass the state test in March.
D. Three-fourths of the way through a unit test.
E. Students who fail a test on Friday have to come back on Saturday.
F. Exit pass question: “What is the difference between mass and weight?”
G. “Sketch the graph of y equals one over one plus x squared on your mini-white boards.”
![Page 18: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/18.jpg)
An assessment functions formatively to the extent that evidence about student achievement is elicited, interpreted, and used by teachers, learners, or their peers, to make decisions about the next steps in instruction that are likely to be better, or better founded, than the decisions they would have taken in the absence of the evidence that was elicited.
Formative Assessment:An Inclusive Definition
![Page 19: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/19.jpg)
Mapping Out the Terrain
Annual
Interim
Weekly
Daily
Instructional Guidance
(“formative”)
Describing Individuals
(“summative”)
Institutional Accountability (“evaluative”)
FunctionFunction
Hourly
High-stakes accountability
Academic promotion
Hinge-pointquestions
End-of-unit tests
Benchmark
Exit pass
Common formative assessments
End-of-course exams
Before the end-of-unit tests
Growth
![Page 20: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/20.jpg)
Sheena leaves a wooden block, a glass flask, a woolly hat, and a metal stapler on a table overnight. What can she say about their temperatures the next morning?
A. The stapler will be colder than the other objectsB. The woolly hat will be warmer than the other objectsC. The temperatures of all four objects will be differentD. The temperatures of all four objects will be the same
Hinge-point questions
![Page 21: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/21.jpg)
Development of science skills in eighth grade– Use of laboratory equipment– Metric unit conversion– Density calculations– Density applications– Density as a characteristic property– Phases of matter– Gas laws– Communication (graphing)– Communication (lab reports)– Inquiry skills
Connecting (some of) the dots
![Page 22: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/22.jpg)
Assessment matrix
![Page 23: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/23.jpg)
![Page 24: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/24.jpg)
• Students can gauge their growth relative to normative or aspirational goals.
• Parents can assess their child’s growth.
• Teachers can use results to make instructional decisions for the current cohort.
• School administrators can aggregate student results to assess trends in growth and to evaluate teacher effectiveness.
Stakeholder Needs
![Page 25: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/25.jpg)
• Teachers: different teachers may be acting effectively in their own classrooms, but their judgments may differ from those of their colleagues
• School administrators need to know that achievement scores are comparable from class to class.
• Parents need assurance that the grades reported to them are meaningful.
Unmet Needs
![Page 26: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/26.jpg)
• Common assessments allow teachers to compare their judgments about the meaning of standards, and thus align their judgments.
• Although these instruments are often called “common formative assessments” their main use may be in aligning teachers’ judgments for summative purposes.
Connecting (more of) the dots
![Page 27: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/27.jpg)
Teacher’s role Learner’s role
Summative assessment A community of practice in which teachers share a construct of quality
Understanding the assessment intentions, so they produce relevant evidence
Formative assessment Teachers possess an anatomy of quality
Learners become members of the same community of practice of which their teachers are already members
![Page 28: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/28.jpg)
• will provide students with immediate feedback concerning their progress
• will provide teachers with information concerning their student’s needs
• will provide teachers with information useful in long-range instructional planning
• will provide school administrators with information about the school’s progress
• will provide the public with information about student achievement and growth
A strong assessment system
![Page 29: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/29.jpg)
• needs to be designed to have an impact in the classroom
• needs to communicate needed information clearly to teachers and students
• needs a strong measurement scale to measure growth
• needs normative, criterion, and content references to make meaning of performance
• needs a strong measurement design to measure growth well
A strong assessment system
![Page 30: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/30.jpg)
• Before school starts, an interim assessment allows accurate class placement, suggests a place to start for each student, and sets individual growth targets.
• During the first term formative assessment processes allow the teacher to adjust the original placement and help each student start to move forward.
• At the end of the first term, an interim assessment gives each teacher and student a first look at achievement during the year and progress toward growth targets.
A combined system
![Page 31: G. Gage Kingsbury, Dylan Wiliam, & Steven L. Wise, NWEA 11 th Annual MARCES/MSDE Conference University of Maryland, October, 2011 Connecting the Dots:](https://reader035.vdocument.in/reader035/viewer/2022062516/56649db25503460f94aa16bd/html5/thumbnails/31.jpg)
• During the second term formative assessment allows each teacher to adjust content as the students progress, and allows regrouping of students using teams of teachers.
• Toward the end of school, the summative assessment identifies the overall achievement of the students in the class to help determine what there is to celebrate, and what might be done better in subsequent years.
• At the end of school, an interim assessment gives each teacher and student a look at achievement during the year and attainment of growth targets.
A combined system continued