![Page 1: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/1.jpg)
Final Reports from the Measures of Effective Teaching ProjectTom KaneHarvard University
Steve Cantrell, Bill & Melinda Gates Foundation
![Page 2: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/2.jpg)
2
The MET project is unique … in the variety of indicators tested,
5 instruments for classroom observations
Student surveys (Tripod Survey)
Value-added on state tests
in its scale,3,000 teachers
22,500 observation scores (7,500 lesson videos x 3 scores)
900 + trained observers
44,500 students completing surveys and supplemental assessments in year 1
3,120 additional observations by principals/peer observers in Hillsborough County, FL
and in the variety of student outcomes studied. Gains on state math and ELA tests
Gains on supplemental tests (BAM & SAT9 OE)
Student-reported outcomes (effort and enjoyment in class)
![Page 3: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/3.jpg)
3
Two Past Reports: Learning about Teaching
(Student Surveys)
Gathering Feedback for Teaching (Classroom Observations)
![Page 4: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/4.jpg)
4
Have we identified effective teachers or … teachers with exceptional students?
To find out, we randomly assigned classrooms to 1,591 teachers.
![Page 5: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/5.jpg)
5
Have We Identified Effective Teachers?KEY FINDINGS
Following random assignment in Year 2, the teachers with greater
measured effectiveness in Year 1 did produce higher student achievement.
The magnitude of the impacts were consistent with predictions.
They also produced higher achievement on the supplemental assessments 70 percent as large as impacts on state tests.
![Page 6: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/6.jpg)
6
![Page 7: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/7.jpg)
7
![Page 8: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/8.jpg)
Organizing Observations by School PersonnelKEY FINDINGS
Adding an observation by a second observer increases reliability twice as much as having the same observer score an additional lesson.
Short observations provide a time-efficient way to incorporate more than one observer per teacher.
School administrators rate their own teachers higher than do outside observers. However, (1) they rank their teachers similarly to others and (2) they discern bigger differences between teachers than peers do (which increases reliability).
Although average scores are higher across the board, letting teachers choose which lessons are observed produces similar rankings and slightly higher reliability.
![Page 9: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/9.jpg)
9
There are many roads to reliability.
![Page 10: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/10.jpg)
10
Combining Measures Using WeightsKEY FINDINGS
The best way to identify teachers who produce large student achievement gains on state tests is to put 65 to 90 percent of the weight on teacher’s past history of gains on such tests. However, the resulting composite does not predict student achievement gains on more cognitively challenging assessments as well.
Balanced weights have somewhat less predictive power with respect to state achievement gains but, they offer (1) better ability to predict other outcomes and (2) improved reliability (less volatility).
It is possible to go too far. Weighting state tests less than one-third results in (1) worse predictive power with respect to other outcomes and (2) less reliability.
![Page 11: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/11.jpg)
11
What’s the best we could do with master’s degrees and experience alone?
Higher Order
.14
State
.13
![Page 12: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/12.jpg)
12
![Page 13: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/13.jpg)
Feedback for Better Teaching
13
January 2013
Steve Cantrell, Bill & Melinda Gates Foundation
![Page 14: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/14.jpg)
14
“MOM P.”
![Page 15: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/15.jpg)
15
Monitor validityEnsure reliabilityAssure accuracy
Make meaningful distinctionsPrioritize support and feedbackUse data for decisions at all levels
Set expectations Use multiple measuresBalance weights
![Page 16: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/16.jpg)
16
Set expectations Use multiple measures Balance weights
![Page 17: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/17.jpg)
17
Monitor validity
Set expectations Use multiple measures Balance weights
![Page 18: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/18.jpg)
18
Monitor validity Ensure reliability
Set expectations Use multiple measures Balance weights
![Page 19: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/19.jpg)
19
Monitor validity Ensure reliability Assure accuracy
Set expectations Use multiple measures Balance weights
![Page 20: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/20.jpg)
20
Make meaningful distinctions Prioritize support and feedback Use data for decisions at all levels
Set expectations Use multiple measures Balance weights
Monitor validity Ensure reliability Assure accuracy
![Page 21: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/21.jpg)
21
Actual scores for 7500 lessons.
Framework for Teaching (Danielson)Unsatisfactory
Yes/no questions; posed in rapid succession; teacher asks all questions; same few students participate.
Basic
Some questions ask for explanations; uneven attempts to engage all students.
Proficient
Most questions ask for explanation; discussion develops, teacher steps aside; all students participate.
Advanced
All questions high quality; students initiate some questions; students engage other students.
![Page 22: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/22.jpg)
22
Set expectations Use multiple measures Balance weights
Monitor validity Ensure reliability Assure accuracy
Make meaningful distinctions Prioritize support and feedback Use data for decisions at all levels
![Page 23: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/23.jpg)
23
![Page 24: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/24.jpg)
24
![Page 25: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/25.jpg)
25
![Page 26: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/26.jpg)
26
![Page 27: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/27.jpg)
27
Achievement Gains
![Page 28: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/28.jpg)
28
Achievement Gains
2009 Average Performance
Below Above
![Page 29: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/29.jpg)
29
Achievement Gains
2009 Average Performance
Below Above
At 2010 PredictedPerformance
Below
Above
![Page 30: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/30.jpg)
30
Achievement Gains
2009 Average Performance
Below Above
Below
Above
Very Low Prior Student Achievement
![Page 31: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/31.jpg)
31
Student Achievement
2009 Average Performance
Below Above
Below
Above
Very Low Prior Student Achievement
Almost All Performing at or above Prediction
![Page 32: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/32.jpg)
32
Classroom Observation
![Page 33: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/33.jpg)
33
Student Surveys
![Page 34: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/34.jpg)
34
![Page 35: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/35.jpg)
35
The Library of Practice
MET Longitudinal Database
Professional Development Studies
Working with Key Partners to Implement Feedback and Evaluation Systems
This Symposium and Your Good Work!
![Page 36: Final Reports from the Measures of Effective Teaching Project Tom Kane Harvard University Steve Cantrell, Bill & Melinda Gates Foundation](https://reader038.vdocument.in/reader038/viewer/2022110101/56649eef5503460f94bff222/html5/thumbnails/36.jpg)
36
You can find this slide presentation, the current reports,
and all past reports at
www.metproject.org