Copyright © 2011

American Institutes

for Research

All rights reserved.

Pilot Implementation Data: What We’ve Learned So Far

Gretchen WeberDirector of Educator Effectiveness Initiatives

January 24, 2012

2

Teacher Evaluation Hopes and Worries: September 2011• 72% of teachers, 83.9% of principals, and 100% of

administrators were moderately to very excited about the new system.

• The top aspects of excitement: Professional growth/development Discussions with administrators Clearly defined, evidence-based standards

• 23.9% of teachers, 47.6% of principals, and 50% of administrators were moderately to very worried about the new system.

• The top worries: Time Fidelity Buy-in

3

Principal Evaluation Hopes and Worries: September 2011

• 75% of principals and 85.7% of administrators were moderately to very excited about the new system.

• Top aspects of excitement: Professional growth/development Clearly defined, evidence-based standards

• 26.9% of principals and 38.5% of administrators were moderately to very worried about the new system.

• Top worry: Time

4

TPEP Case Studies: Methodology

• Self-Report Data Professional Development Resources

• Surveys Teachers Principals Administrators

• Interviews Teachers Principals Pilot leads Union/association representative

5

TPEP Case Studies: Constructs

• The new evaluation framework and goals

• Performance measures • Professional development• Communication• Time and resources for implementation • District and school culture changes

6

TPEP Case Studies: Cross-Case Analysis

• Each of the site-specific case study reports was reviewed.

• Major themes across sites were recorded.

• Survey responses from teachers, principals, and administrators across sites were produced to indicate areas of strength and weakness.

• The report represents the total sample.

7

TPEP Case Studies: Overall Findings on Teacher Evaluation

• The vast majority of teacher respondents in the pilot reported understanding the new system—its goals, measures, and how to achieve a high rating.

• They reported that the system is useful and helpful to professional growth.

• They see the new teacher evaluation system as better than the previous system in helping them improve their practice.

• Simultaneously teacher respondents raised concerns about the new system, particularly the time involved in evaluation activities but also communication with non-pilot teachers and the extent to which the system can account for differences in teachers’ content areas or student populations.

8

TPEP Case Studies: Overall Findings on Principal Evaluation

• The development of principal evaluation systems is at various stages in the pilots. While interviewees in a couple districts described a well-specified principal evaluation system, many interviewees in other districts suggest that the development of the system is just beginning.

• The vast majority of principals report understanding the goals and measures of the new principal evaluation system.

• A strong majority report that it is better than the previous system in terms of helping to improve their practice.

• Similarly to the new teacher evaluation system, principals raised concerns about the amount of time involved and communication with non-pilot principals.

9

TPEP Case Studies: General Findings

• The new evaluation system takes more time and money to understand and implement than the previous one. Over 760 total session hours have been spent formally training

educators on the new system thus far (through September 30, 2011).

64.4% of teachers reported spending a much greater amount of time on evaluation activities than previously. 

65.8% of principals spent more than 10 hours on evaluation activities per teacher in the new evaluation system thus far.

In total, 57.9% of administrators reported spending 21 or more hours on evaluation activities for principals.

Only 47.8% of administrators believe there are adequate monetary resources to support the implementation of the new evaluation systems.

10

TPEP Case Studies: General Findings

• The investment may be worth it. The practice of educators, already of high quality in Washington, appears to be improving even more. The measures being used in the new teacher evaluation system

provide feedback for improving instructional practice. More than 88% of teachers and principals agree.

The system supports professional growth. 75.4% of teachers consult the instructional framework to inform their professional practice. 45.8% of principals who report having a leadership framework consult the framework to inform their practice.

Over 84% of teachers and principals agree that the measures in the teacher evaluation system are fair, incorporate the most important aspects, and provide a comprehensive picture of teacher performance. 

84.7% of principals indicate that their system provides useful feedback to improve their practice as a leader.

11

TPEP Case Studies: General Findings

• Over the long-term the investment should continue to pay off. A shift in the culture across schools and districts appears to be happening and the work of the pilots is just the start. 74.6% of teachers agree that the teacher evaluation system is more

objective than before. Almost 90% of teachers and principals indicate that that the new

systems specifies how to improve, identifies areas of strength, and encourages better teaching. 

70.8% of principals and 90.9% of administrators agree that the new principal evaluation system is better than the old one to improve leadership practice. 

78.3% of principals and 100.0% of administrators report more productive conversations due to new principal evaluation system.

12

TPEP Case Studies: Next Steps

• Spring data collection Self-report data (time and resources) Surveys Document review

• Final case studies and cross-case analysis

13

Gretchen Weber

P: 630-649-6511

E-Mail: gweber@air.org

1120 East Diehl Road, Suite 200

Naperville, IL 60563-1486

General Information: 800-356-2735

Website: www.air.org