Measuring Up: Effective Strategies for Teacher

EvaluationPatricia A. Popp, Ph.D.

State Coordinator, Project HOPE-VAClinical Associate Professor

pxpopp@wm.edu

Xianxuan Xu, Ph.D.Post-Doctoral Research Associate

xxu@wm.edu

School of EducationThe College of William and Mary

Teacher Evaluation in an Era of Accountability

• Currently, ESEA flexibility has been granted to 34 states and the District of Columbia. As part of the flexibility requirements, the states were required to establish new teacher evaluation systems that factor in student achievement progress for statewide implementation by the end of the 2013-2014 school year.

• All states that received Race to the Top funding are undertaking substantial reforms with teacher evaluation.

• “The quality of an education system cannot exceed the quality of its teachers” (Barber & Mourshed, 2008).

Share of At-Risk Students on PISA(Not reaching PISA baselines): Reading

Canada Finland Shanghai Singapore South Korea

United States

0

5

10

15

20

25

118

5

108

21

Share of At-Risk Students on PISA(Not reaching PISA baselines): Math

Canad

a

Finlan

d

Shan

ghai

Singa

pore

South

Korea

United

State

s05

10152025

118

510 8

21

Percentage of Country’s Students in PISA Top Performing Groups:

Reading

048

121620

1315

1916

1310

Percentage of Country’s Students in PISA Top Performing Groups:

Math

Canad

a

Finlan

d

Shan

ghai

Singa

pre

South

Korea

United

State

s0

102030405060

18 21

50

3626

10

What do we do about it?

How Long Do Students Attend School? Days in the Academic YearCountry Days in an

Academic YearCanada Average: 188Finland 187

Singapore 200Shanghai 180

South Korea 204United States 180

How Long Do Students Attend Schools?Minutes in the Day

Location Minutes in a School Day

Canada 304Finland 240

Shanghai 390Singapore 330

South Korea 264United States 402

How Much Do We Spend?

Location Annual Expenditures Per PupilCanada 8,045Finland 7,216

Shanghai N/ASingapore N/A

South Korea

6,663

United States

10,259

Student/Teacher Ratio

CountryAverage

Student/Teacher Ratio

Canada 25Finland 19

Shanghai 39Singapore 35

South Korea 36United States 24

Source: Available at worldbank.org.

Which factor is a strong predictor of student achievement gains?

Sources: Wright, Horn, & Sanders, 1997;Hanushek, Kain, & Rivkin, 1998,

Class sizeClassroom heterogeneitySchool resource differencesIt’s the teacher.

Influences on Student Achievement:Explained Variance

Students50%

Peers5-10%

School5-10%

Home5-10%

Teachers30%

Source: Hattie, J. Teachers make a difference: What is the research evidence.Retrieved 20Nov08 from http://acer.edu.au/documents

Dallas Research: Teacher Quality

1st

grad

e

4th

grad

e0

20

40

60

80

1004th gr. Math Achievement

Highly Effective

IneffectiveDallas, Texas data: 2800-3200 students per cohort

Comparison of 3 “highly effective” & 3 “ineffective” teachers (Jordan, Mendro, & Weerasinghe, 1997)

Dallas Research: Teacher Quality

1st G

...

4th

...

010203040506070

4th gr. Reading Achievement

Highly Effective

Ineffective

Time in School Year Needed to Achieve the Same Amount of Learning

Leigh, Economics of Education Review (2010)

0 1/4 1/2 3/4 1

25th PercentileTeacher

75th PercentileTeacher

Years Needed

Time in School Year Needed to Achieve the Same Amount of Learning

0 1/4 1/2 3/ 4 1

10th PercentileTeacher

90th PercentileTeacher

Years Needed

Source: Leigh, A. (n.d.). Estimating teacher effectiveness from two-year changes in students’ test scores. Retrieved May 22, 2007, from http://econrsss.anu.edu.au/~aleigh/.

Time in School Needed to Achieve the Same Amount of Learning

Sequence of Effective Teachers

Low

High

52-54percentile

points difference

Low Low

High High

Sanders & Rivers (1996)

Sequence of Effective Teachers

Low

High

13percentile

points difference

Low

High High

Sanders & Rivers (1996)

High

Revised Teacher Evaluation System

in Virginia: An Overview

Primary Purposes of the Evaluation System

• Improve student achievement through the quality of instruction by assuring accountability for classroom performance

• Contribute to the successful achievement of the goals and objectives defined in a school division’s educational plans

• Provide a basis for instructional improvement through productive teacher appraisal and professional growth

• Share responsibility for evaluation between the teacher and the evaluation team in a collaborative process that promotes self-growth, instructional effectiveness, and improvement of overall job performance

A Flawed System

Problem No. 1: Observation equals evaluation Problem No. 2: Likely to rely on intuition, not evidence, to

make judgments about teacher performanceProblem No. 3: One size fits allProblem No. 4: Don’t communicateProblem No. 5: Fragmented evaluation processProblem No. 6: Irrelevant evaluationProblem No. 7: One-point rating scalesProblem No. 8: No impact evaluation

Question 1

What is the basis of the teachers’ evaluation?

Standard 2: Instructional PlanningThe teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the learning needs of all students.

Main Components

Sample Performance IndicatorsExamples may include, but are not limited to:

The teacher: 2.1 Uses student learning data to guide planning.2.2 Plans time realistically for pacing, content mastery, and transitions.2.3 Plans for differentiated instruction.2.4 Aligns lesson objectives to the school’s curriculum and student learning needs.2.5 Develops appropriate long- and short-range plans, and adapts plans when needed.

ExemplaryProficient

Proficient is the expected level of performance.

Developing/Needs Improvement Unacceptable

In addition to meeting the standard, the teacher actively seeks and uses alternative data and resources and consistently differentiates plans to meet the needs of all students.

The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.

The teacher inconsistently uses the school’s curriculum, effective strategies, resources, and data in planning to meet the needs of all students.

The teacher does not plan, or plans without adequately using the school’s curriculum, effective strategies, resources, and data.

Performance Appraisal

Rubric

Performance Standard

Performance Indicators

Performance Standards

Professional

Knowledge

Instructional

Planning

Instructional

Delivery

Assessment of

and for Student Learnin

gLearning

Environment

ProfessionalismStudent

Academic

Progress

The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.

Teacher Performance Standard 1: Professional Knowledge

Teacher Performance Standard 2: Instructional Planning

The teacher plans using the Virginia Standards of Learning, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.

Teacher Performance Standard 3: Instructional Delivery

The teacher effectively engages students in learning by using a variety of instructional strategies in order to meet individual learning needs.

Teacher Performance Standard 4: Assessment of and for Student Learning

The teacher systematically gathers, analyzes, and uses all relevant data to measure student academic progress, guide instructional content and delivery methods, and provide timely feedback to both students and parents throughout the school year.

Teacher Performance Standard 5: Learning Environment

The teacher uses resources, routines, and procedures to provide a respectful, positive, safe, student-centered environment that is conducive to learning.

Teacher Performance Standard 6: Professionalism

The teacher maintains a commitment to professional ethics, communicates effectively, and takes responsibility for and participates in professional growth that results in enhanced student learning.

Teacher Performance Standard 7: Student Academic Progress

The work of the teacher results in acceptable, measurable, and appropriate student academic progress.

Question 2

How will teacher performance be documented?

Multiple Data Sources

Data Source Evaluator TeacherInformal Observations Formal Observations Student Surveys Portfolios/Document Logs Self-Evaluation Measures of Academic Progress Reviews/approves Selects/develops

Measures of Academic Progress

TeachersPercentage of

Evaluation based on Student Growth

Percentiles (SGPs)

Percentage of Evaluation based on

Other Growth Measures

Teachers of reading and mathematics for whom SGPs are available

20 20

Teachers who support instruction in reading and mathematics for whom SGPs are available

No more than 20 20 to 40

Teachers who have no direct or indirect role in teaching reading or mathematics in grades where SGPs are available

N/A 40

Student Achievement Goal Setting

Step 1:Determine

needs

Step 2:Create specific learning

goals based on pre-

assessment

Step 3: Create and implement

teaching and learning

strategies

Step 4: Monitor student progress through ongoing

formative assessment

Step 5:Determine

whether the students

achieved the goal

What are the Purposes ofStudent Achievement Goal Setting?

Focus on student results Explicitly connect teaching and

learning Improve instructional practices and

teacher performance Tool for school improvement

Question 3

How will teacher performance be

rated?

Evaluations

Interim Evaluation • Used to document evidence of meeting standards• Does NOT include rating of performance

Summative Evaluation• Comes at end of evaluation cycle

- One year for probationary teachers- Three years for continuing contract teachers

• Assessment of performance quality- Four point rating scale- Performance rubric for every standard

Evaluating PerformanceExemplary The teacher maintains

performance, accomplishments, and behaviors that consistently and considerably surpass the established standard.

Exceptional Performance• Sustains high performance over period of time• Behaviors have strong positive impact on

learners and school climate• Serves as role model to others

Category Description Definition

Proficient The teacher meets the standard in a manner that is consistent with the school’s mission and goals.

Effective Performance• Meets the requirements contained in job

description as expressed in evaluation criteria• Behaviors have positive impact on learners and

school climate• Willing to learn and apply new skills

Developing/Needs Improvement

The teacher often performs below the established standard or in a manner that is inconsistent with the school’s missions and goals.

Below Acceptable Performance• Requires support in meeting the standards• Results in less than quality work performance• Leads to areas for teacher improvement being

jointly identified and planned between teacher and evaluator

Unacceptable The teacher consistently performs below the established standards or in a manner that is inconsistent with the school’s missions and goals.

Ineffective Performance• Does not meet requirements contained in job

description as expressed in evaluation criteria• Results in minimal student learning• May result in employee not being

recommended for continued employment

Sample Performance Appraisal Rubric

ExemplaryProficient

Proficient is the expected level of performance.

Developing/Needs Improvement Unacceptable

In addition to meeting the standard, the teacher consistently demonstrates extensive knowledge of the subject matter and continually enriches the curriculum.

The teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.

The teacher inconsistently demonstrates understanding of the curriculum, content, and student development or lacks fluidity in using the knowledge in practice.

The teacher bases instruction on material that is inaccurate or out-of-date and/or inadequately addresses the developmental needs of students.

Standard I: Professional KnowledgeThe teacher demonstrates an understanding of the curriculum, subject content, and the developmental needs of students by providing relevant learning experiences.

Summative ratings should apply the rating for each of the seven performance expectations, with the most significant weight given to Standard 7 - Student Academic Progress.

Summative Rating

• Weight each of the first six standards equally at 10 percent each

• Weight Standard 7 – Student Academic Progress at 40 percent

Lessons Learned

Most practitioners believe new models provide specific measures of teacher effectiveness that are useful for distinguishing effective from less effective teachers.

ExemplaryProficient

Proficient is the expected level of performance.

Developing/Needs Improvement Unacceptable

In addition to meeting the standard, the teacher actively seeks and uses alternative data and resources and consistently differentiates plans to meet the needs of all students.

The teacher plans using the state’s standards, the school’s curriculum, effective strategies, resources, and data to meet the needs of all students.

The teacher inconsistently uses the school’s curriculum, effective strategies, resources, and data in planning to meet the needs of all students.

The teacher does not plan, or plans without adequately using the school’s curriculum, effective strategies, resources, and data.

Most practitioners believe that new models have the potential to improve teaching and learning by providing useful feedback that can be used to diagnose and guide teacher improvement.

Formal Observation Post-Conference

Student Learning Objectives

Teacher Self-Assessment

Student Surveys

Lessons Learned

Lessons Learned

With more rigorous assessment to identify problems and recognize excellence, investments in teacher development can be better related to school and division goals for improvement.

Evaluation

Strengths and

Weaknesses Identified

Targeted Professional Developme

nt

Lessons Learned

Practitioners appreciate the value in using multiple data sources to provide evidence of performance standards.

Teacher Performance

Standards

Observations

Documentation

Logs

Student Learning

Objectives

Student Surveys

Lessons Learned

Practitioners believe that new models set up realistic expectations for teacher performance, and they reflect the most important elements of effective teaching.

Professional Knowledge

Instructional Planning

Assessment of/forLearning

Instructional Delivery Learning Environment

Professionalism

Student Progress

Lessons Learned

The evaluation framework is valid in terms that the process standard ratings of teacher have a moderate ability to predict the student academic progress. In addition, there is a significant correlation between each of the six process standards and student academic progress.

Lessons Learned

New models can be time-consuming to implement.

Practitioners mistrust the validity of student progress models as a measure of student growth.

Teachers believe that the nature, quality, and credibility of the evaluation process vary depending on the qualifications of the evaluators.

Concluding Thoughts on Transforming Teacher Evaluation

State• Convene stakeholders across the state to design,

implement, and improve evaluation system.• Develop validated and reliable evaluation measures.• Provide incentives and on-going support to school

divisions.• Make knowledge of new developments in teacher

evaluation part of leadership and teacher preparation programs.

• Make evaluation count.

Concluding Thoughts on Transforming Teacher Evaluation (Continued)

Schools and School Divisions/Districts• Clear expectations of the WHAT will be evaluated.• Clear communication of the evaluative criteria for

effective performance.• Instruments and procedures for teachers to

provide evidence from multiple sources of the HOW of their performance.

• Increase the use of evaluation results to inform professional development.