impacts of comprehensive teacher induction: final results from a randomized trial ies summer...
TRANSCRIPT
Impacts of Comprehensive Teacher Induction: Final Results
from a Randomized Trial
IES Summer Research Conference, June 2010
Steven Glazerman ● Eric Isenberg ● Sarah Dolfin ● Martha Bleeker Amy Johnson ● Mary Grider ● Matthew Jacobus
2
Mentors– Carefully selected and trained
– Full-time release with ratio of 12:1
Curriculum– Instructionally focused
– Structured and sequenced
Activities– Weekly meetings with mentor, written logs
– Monthly study groups
– Classroom observation with formative assessment
– End-of-year colloquium
– Coordination with administrators and program staff
What is “Comprehensive Induction”?
2
3
Compared to prevailing induction, what is the impact of comprehensive induction on…Induction services?
– Whether assigned a mentor– Time spent with mentor– Activities
Workforce outcomes?– Teacher attitudes– Teacher retention
Classroom outcomes?– Teacher practices– Student test scores
Research Questions
3
4
Selected 17 districts Randomized 418 elementary
schools Followed 1,009 teachers
– 698 eligible for classroom observation– 190 eligible for test-score analysis in year 3
In second year of study, created two experiments– “One-year districts” with a single year
of treatment– “Two-year districts” with two years
of treatment
Study Design
4
5
Induction services– Control group received induction services
– Treatment group received more during intervention period
Workforce outcomes– No impact on attitudes
– No impact on teacher retention, mobility
Classroom outcomes– No impacts on classroom practices in first year
– No impacts on test scores in one-year districts
– Positive impacts on test scores in two-year districts
• Years 1 and 2: no impacts
• Year 3: Effect size = 0.11 (reading) and 0.20 (math)
• Positive impacts sensitive to sample definition
Summary of Findings
5
Induction Support
7
Time Spent with Mentors: One-Year Districts
7
Solid squares = Treatment-control difference is statistically significant at the 0.05 level.
8
Time Spent with Mentors: One-Year Districts
8
Solid squares = Treatment-control difference is statistically significant at the 0.05 level.
Intervention period
9
Time Spent with Mentors: One-Year Districts
9
Solid squares = Treatment-control difference is statistically significant at the 0.05 level.
Intervention period
10
Time Spent with Mentors: Two-Year Districts
10
Solid squares = Treatment-control difference is statistically significant at the 0.05 level.
11
Time Spent with Mentors: Two-Year Districts
11
Solid squares = Treatment-control difference is statistically significant at the 0.05 level.
Intervention period
Impacts on the Workforce:Teacher Mobility
13
Retention in the District: One-Year Districts
13
Note: No impacts are significantly different from zero at the 0.05 level.
14
Retention in the District: Two-Year Districts
14
Note: No impacts are significantly different from zero at the 0.05 level.
Percent
Impacts on the Classroom:Student Achievement
Impacts on Test Scores, Year 3
16
*Treatment-control difference is statistically significant at the 0.05 level.
17
Sensitivity Tests, Reading in Two-Year Districts
17
*Significantly different from zero at the 0.05 level.
Model
Impact (Effect Size)
Standard Error
Sample Size
(Teachers)
1. Benchmark 0.11* 0.05 74
2. Drop data restrictions 0.11* 0.05 74
3. Allow comparisons across grades 0.16* 0.05 82
4. Drop pretest, benchmark sample 0.05 0.08 74
5. Drop pretest, expanded sample -0.07 0.09 127
18
Sensitivity Tests, Math in Two-Year Districts
18
*Significantly different from zero at the 0.05 level.
Model
Impact (Effect Size)
Standard Error
Sample Size
(Teachers)
1. Benchmark 0.20* 0.05 68
2. Drop data restrictions 0.23* 0.05 70
3. Allow comparisons across grades 0.13* 0.06 77
4. Drop pretest, benchmark sample 0.15 0.08 68
5. Drop pretest, expanded sample -0.03 0.09 120
19
Induction services– Control group received induction services
– Treatment group received more during intervention period
Workforce outcomes– No impact on attitudes
– No impact on teacher retention, mobility
Classroom outcomes– No impacts on classroom practices in first year
– No impacts on test scores in one-year districts
– Positive impacts on test scores in two-year districts
• Years 1 and 2: no impacts
• Year 3: Effect size = 0.11 (reading) and 0.20 (math)
• Positive impacts sensitive to sample definition
Summary of Findings
19
20
Please contact– Steven Glazerman
Report is available online at:– http://ies.ed.gov/ncee/pubs/20104027/
For More Information
20
END
(extra slides follow)
21
22
Percent With a Mentor Assigned: One-Year Districts
22
Solid squares = Treatment-control difference is statistically significant at the 0.05 level.
Intervention period
23
Percent with a Mentor Assigned:Two-Year Districts
23
Solid squares = Treatment-control difference is statistically significant at the 0.05 level.
Intervention period
24
Retention in Teaching: One-Year Districts
24
Note: No impacts are significantly different from zero at the 0.05 level.
25
Retention in Teaching: Two-Year Districts
25
Note: No impacts are significantly different from zero at the 0.05 level.
Impacts on the Workforce:Teacher Attitudes
27
No significant impacts on satisfaction with—
– Career
– Class
– School
No significant impacts on feelings of preparedness to—
– Instruct
– Work with others
– Work with students
No Impacts on Teacher Attitudes
27
28
No Composition Effects
28
Treatment stayers vs. control stayers
Findings
– Professional characteristics of teachers:
no difference
– Classroom practices in year 1: no positive impact
– Student achievement in year 3: no positive impact
Impacts on the Classroom:Teacher Practices
30
No Impact on Year 1 Classroom Practices
30
Noevidence
Consistentevidence
Limitedevidence
Extensive evidence
Moderateevidence
Treatment-control differences are not statistically significant (N = 631 teachers).
31
Sensitivity Tests, Reading in One-Year Districts
31
Treatment-control differences are not statistically significant.
• Model
Impact (Effect Size)
Standard Error
Sample Size
(Teachers)
1. Benchmark 0.01 0.04 99
2. Drop data restrictions 0.02 0.03 107
3. Allow comparisons across grades 0.01 0.05 114
4. Drop pretest, benchmark sample 0.10 0.06 99
5. Drop pretest, expanded sample 0.10 0.06 151
32
Sensitivity Tests, Math in One-Year Districts
32
*Significantly different from zero at the 0.05 level.
• Model
Impact (Effect Size)
Standard Error
Sample Size
(Teachers)
1. Benchmark -0.10 0.06 95
2. Drop data restrictions -0.12* 0.06 97
3. Allow comparisons across grades -0.07 0.06 104
4. Drop pretest, benchmark sample 0.03 0.09 95
5. Drop pretest, expanded sample 0.08 0.07 138