© TNTP 2013

Introduction to ACEAugust 2014

For ACE Observers and TNTP Instructional Staff

/ 2

Objectives

By the end of this session, you will:

Understand the purpose of assessing teacher effectiveness

Understand each measure used in the Assessment of Classroom Effectiveness (ACE)

Understand the process and tools used in ACE Observations

Understand ACE scoring and potential outcomes

1

2

3

4

/ 3

Agenda

Why We Care About Effective Teaching

Understanding the Assessment of Classroom Effectiveness (ACE)

Adding It Up: ACE Scoring

/ 4

By the end of high school, African-American and Hispanic students read and do math at virtually the same level as eighth-grade white students.

On the 2011 National Assessment of Educational Progress (NAEP), only 14% of African-American eighth-graders and 21% of Hispanic eighth graders scored at the proficient or advanced levels in math.

In urban areas, half of students won’t graduate at all, closing the door to a college degree or a middle-class living.

In a nation dedicated to equality and opportunity for all, poor and minority students still receive an unequal education, undermining the hopes of millions to compete for a better life.

Nationwide, poor and minority students receive a second-rate education, with unequal access to the resource that matters most: great teachers.

/ 5

Even one year with a highly effective teacher has a lifelong impact on students.

Great teaching changes lives.

/ 6

Agenda

Why We Care About Effective Teaching

Understanding the Assessment of Classroom Effectiveness (ACE)

Adding It Up: ACE Scoring

/ 7

Future teachers join us because they want to make a real difference for high-need students. We use ACE to help them become the best teachers they can be.

We are committed to helping teachers make a difference from day one, but we have a responsibility to ensure that they remain on track to become great teachers.

TNTP Academy uses the Assessment of Classroom Effectiveness (ACE) to create the fullest possible picture of all teachers’ performance using multiple measures.

ACE is more than an evaluation system; it is designed to support rapid growth by ensuring that teachers always know how they’re doing and what they need to do to improve.

We use ACE not only to assess teacher growth but to customize our training to meet unique developmental needs.

At the end of the year, teachers must pass our ACE screen in order to be recommended for certification. Earning certification under ACE is a high bar and an honor.

/ 8

Through ACE, we strive to create the fullest possible picture of our teachers’ performance using multiple measures.

• Research shows classroom observations are valid predictors of teacher effectiveness.

• Teachers will be observed at least 4 times, with performance rated according to the ACE Instructional Framework.

Classroom Observatio

ns

• Principals will rate teacher performance relative to other new teachers they have worked with throughout their careers.

Principal Ratings

• The survey uses research-tested questions predictive of teacher effectiveness.

• If teachers work with very young students (grades K-2) or with students in moderate-to-severe special education settings, they will not administer student surveys.

Student Surveys

• The best measure of effectiveness is the academic growth of one’s students.

• Where available, we will incorporate data from state-produced value-added reports to analyze teachers’ ;actual impact on student learning over the course of the school year.

Student Achieveme

nt Data

/ 9

ACE helps teachers be more successful by providing regular insights about their practice.

Taylor, E. & Tyler, J. (2011). The effect of evaluation on performance: Evidence from longitudinal student achievement data of mid-career teachers. NBER Working Paper # 16877. Cambridge, MA: National Bureau for Economic Research.

Impact of Teacher Evalu-ation on Student Math

Scores

Pre-evaluationPost-evaluation

• Teachers receive at least 4 ACE observations annually and get detailed reports shortly thereafter.

• Student Survey reports are released in early spring, so teachers can respond to that feedback within the year.

• We use ACE data to customize training and coaching to address your high-priority development needs

• We share ACE data with principals to help teachers access relevant PD resources through their school and district.

Personalized Support

Research shows that teachers improve when they receive

quality feedback through evaluations.

/ 10

Measure 1: Classroom Observations

WHY?• Research shows that

rigorous observations of teachers at work are valid predictors of teacher effectiveness.

• Observation ratings and feedback can give you useful insights on your strengths and weaknesses.

HOW?• Trained ACE Observers

will use our ACE Instructional Framework to evaluate teacher performance. They will be looking for hallmarks of effective teaching that boost student learning.

• Each observation will be scored in 4 performance areas, producing an average score for the observation session.

• Teachers should receive a report shortly after each observation.

WHEN?• Teachers will receive at

least 4 ACE observations, starting in October/November.

• Observations will last 45-60 minutes or one full class period.

• All observations will be announced.

• Each month, teachers will need to input a copy of their teaching schedule into our online tracking system so we can schedule observations.

NOTE: ACE observations are

separate from non-evaluative observations conducted by

Academy staff or any observations

conducted by your

district/school-based mentors or

observers.

/ 11

After each observation, teachers will get a report showing their scores in four performance areas, along with an overall observation score.

PERFORMANCE AREA SCORE

Student Engagement 3

Essential Content 2

Academic Ownership 2

Demonstration of Learning 3

OVERALL SCORE FOR THIS OBSERVATION: 2.50

ACE observation scores will be calculated by averaging the competency scores for each competency, with the overall scores aligning to the performance levels below.

Skillful Proficient Developing

Minimally Effective

Ineffective

4.30 – 5.00 3.60 – 4.29 2.80 – 3.59 2.00 – 2.79 1.00 – 1.99

/ 12

Measure 2: Principal Evaluation

In the spring, principals will rate teacher performance relative to other first-year teachers he or she has worked with in the past.

Performance will be rated on a 5-point scale:

1. Much worse than other first-year teachers2. Worse than other first-year teachers3. About the same as other first-year teachers4. Better than other first-year teachers5. Much better than other first-year teachers

Measurement for Principal Evaluation

Principal evaluations under ACE are entirely separate from school-based evaluations or

school/district renewal decisions.

/ 13

Measure 3: Student Surveys

WHY?• Research from the

Gates Foundation’s Measures of Effective Teaching (MET) study demonstrates that student surveys are a reliable indicator of teacher effectiveness.

• Our student survey uses questions shown to be predictive of teacher impact on student learning.

• Student surveys give voice to the people who have the most direct experience with teachers – the children in their classrooms every day.

HOW?• Students in grades 3-

12 will take paper-based surveys.

• Teachers working with very young students (K-2) or in moderate-to-severe settings will not administer the survey.

WHEN?• Student surveys will

be administered between February and March 2015.

“The average student knows effective teaching when he or she sees it.”(“Learning about Teaching: Initial Findings from the Measures of Effective Teaching,” Gates Foundation, December 2010)

/ 14

How the student survey process works

Students complete an anonymous, paper-based

survey. The survey administration takes

approximately one class period and will designed to ensure confidentiality.

1. ADMINISTER

SURVEY Survey questions are grouped into three broad categories: Care, Control,

and Challenge, with responses rated on a 5-

point scale.

Student responses will be averaged to generate a

composite score in each of these categories, and those three category scores will then be

averaged to generate the overall survey score.

2. GENERATE SCORE

Survey Score

Teachers receive formative score reports detailing their students’ perceptions. This

data provides valuable insights into your strengths and weaknesses as seen by

the students in your classrooms every day.

3. IMPROVE PERFORMANCE

Student Survey

1 – 5

/ 15

Calculating the survey score.

Teacher’s overall survey scores map to the performance levels below.

Skillful Proficient Developing Minimally Effective Ineffective

4.50 – 5.00 4.00 – 4.49 3.50 – 3.99 3.00 – 3.49 2.99 or lower

  STUDENT COMPOSITE SCORES BY CATEGORY

  Care Control ChallengeHarry Potter 4 4 2.75Hermione Granger 3.75 4 2.5Ronald Weasley 4.25 3.5 3Cho Chang 4 3.75 2.75Neville Longbottom 4.5 4.25 3.25Lavender Brown 3.75 3.75 2.75Padma Patil 4.25 3.25 2.75AVERAGE CATEGORY SCORES 4.07 3.79 2.82

TEACHER’S OVERALL SURVEY SCORE: 3.56

/ 16

Past participants valued student survey feedback, indicating that it would help them improve their teaching.

• “I was glad to see my students recognized the value of the homework I assigned, that they believe that I want them to explain their thinking, and that they knew that I believed that they could get a good grade if they tried.“

CHALLENGE

• “I deeply cared for my students, but I know now that I didn't show them or tell them enough. This is definitely something I will focus on next year.“

CARE

• “[After reviewing survey results,] I am going to give the kids more input into the behavior system in my classroom, in an effort to give them ownership of their own actions as individuals and as a class.”

CONTROL

/ 17

Measure 4: Student Achievement Data

We believe that the best measure of teachers’ effectiveness is the academic growth of their students, and we are committed to using student achievement data in ACE when possible.

Where available, we will use data drawn from state value-add reports to assess teacher impact on student learning.

Value-added models emphasize growth rather than absolute proficiency, recognizing that students arrive with varied levels of preparedness, but all are capable of learning.

Value-added models also account for factors beyond teachers’ control, including students’ past academic performance and student characteristics.

When value-added reports are released by the state, we will use the ratings provided by the state to map teacher performance to our ACE rating scale.

/ 18

Agenda

Why We Care About Effective Teaching

Understanding the Assessment of Classroom Effectiveness (ACE)

Adding It Up: ACE Scoring

/ 19

Performance on each ACE measure will be mapped to the performance scale below, earning between 1 and 5 points for each ACE component.

Each component will be weighted and combined to calculate the final ACE rating.

ClassroomObservations

Principal Ratings

Student Surveys

Student Achievement

Ineffective1.00 – 1.99

points

Much worse than

1 point

2.99 or lower1 point

Ineffective1 point

Minimally Effective

2.00 – 2.79 points

Worse than2 points

3.00 – 3.492 points

Minimally Effective2 points

Developing2.80 – 3.59

pointsSame as3 points

3.50 – 3.993 points

Developing3 points

Proficient3.60 – 4.29

pointsBetter than

4 points4.00 – 4.49

4 pointsProficient4 points

Skillful4.30 – 5.00

points

Much better than

5 points

4.50 – 5.005 points

Skillful5 points

/ 20

While ACE considers multiple measures, those measures carry different weights toward the final ACE score.

Scenario 1:Classroom

Observations

Principal Rating

Scenario 2:Classroom

Observations

Principal Rating

Student Surveys

Scenario 3:Classroom

Observations

Principal Rating

Student Surveys

Student Achievement

Data

The weights for each ACE measure will vary, depending on the total number of measures

available given one’s specific teaching placement.

Scenario 4:Classroom

Observations

Principal Rating

Student Achievement

Data

/ 21

Adding it up: Four Scoring Scenarios

Scenario 1: Classroom Observations + Principal Ratings

Observations

(60%)

PrincipalRating(40%)

0.60 * 3.15 points = 1.89 points (Observations)+ 0.40 * 4 points = 1.60 points (Principal Rating)

3.49 points ( Final ACE Score)

Scenario 2: Classroom Observations + Principal Ratings + Student Surveys

Student

Surveys

(30%)

Observations

(40%) PrincipalRating(30%)

0.40 * 3.15 points = 1.26 points (Classroom Observations)+ 0.30 * 3 points = 0.90 points (Principal Rating)+ 0.30 * 2 points = 0.60 points (Student Surveys)

2.76 points ( Final ACE Score)

/ 22

Adding it up: Four Scoring Scenarios, cont’d

0.25 * 3.15 points = 0.79 points (Observations)+ 0.20 * 3 points = 0.60 points (Principal Rating)+ 0.20 * 2 points = 0.40 points (Student Surveys)+ 0.35 * 4 points = 1.45 points (Achievement)

3.24 points ( Final ACE Score)

StudentAchievem

ent(35%) Principal

Rating(20%)

Observations

(25%)

Student

Surveys

(20%)

Scenario 3: Classroom Observations + Principal Ratings + Student

Surveys + Student Achievement Data

Scenario 4: Classroom Observations + Principal Ratings + Student

Achievement Data

Observations

(35%)

StudentAchievem

ent(35%)

Principal Rating(30%)

0.35 * 3.15 points = 1.10 points (Observations) + 0.35 * 4 points = 1.40 points (Student Achievement) + 0.30 * 3 points = 0.90 points (Principal Rating)

3.4 points ( Final ACE Score)

/ 23

There are 3 possible outcomes at the end of your first year of teaching.

PASS2.75 – 5.00 points

EXTENSION2.50 – 2.74 points

REMOVAL1.00 – 2.49 points

• If teachers pass ACE and complete all program and state requirements, they will be recommended for a Level 1 Certificate.

• If teachers will have value-added data available in the late summer, their score is provisional at this stage. They will be eligible to teach in 2014-15 on a Practitioner License.

• If teachers fall short of passing, but are on the right trajectory to meet that standard with additional progress, they may be granted an extension year to continue building skills.

• If teachers will have value-added data available in the late summer, their score is provisional at this stage. They will be eligible to teach in 2014-15 on a Practitioner License.

• If teachers struggle and show limited prospect of improvement, they will not advance to another year of teaching.

• While we expect this to be rare, it is necessary. Allowing a chronically struggling teacher to remain in the classroom isn’t good for anyone.

• Removed teachers will be able to appeal.

• Pending an appeal, this decision is final, even if teacher is awaiting value-added data.

/ 24

Once value-add reports are released, there are two possible outcomes for teachers who received provisional ratings in the spring.

PASS2.75 points or

higher

EXTENSION2.74 points or

lower• Teachers who pass

ACE and successfully complete all program and state requirements will be recommended for their Level 1 Certificate.

• Passing ACE is not a job guarantee; ACE is separate from principal or district renewal decisions.

• If teachers fall short of passing, but are on the right trajectory to meet that standard with additional progress, they will continue teaching on a Practitioner License and be reassessed during 2014-2015.

• Being extended through ACE is not a job guarantee; ACE is separate from principal or district renewal decisions.

/ 25

Recommended for Certification “with Distinction”

Our experience with ACE has shown that while many teachers get off to a strong start, some are truly exceptional. Through ACE, we aim to celebrate these exemplary new teachers by recommending them for certification “with distinction.”

To be recommended for certification with distinction, teachers must demonstrate strong performance across all various ACE measures, earning a cumulative ACE rating in the top 10% of all TNTP Academy participants in Louisiana.

If teachers meet this standard, they will receive a special certificate recognizing their achievements. TNTP Academy will also notify principals and district officials about their accomplishments as a first-year teacher.  

CELEBRATING EXCEPTIONAL

PERFORMANCE