tntp hisd human capital analysis summary memo 8 16 2010 final

8/8/2019 TNTP HISD Human Capital Analysis Summary Memo 8 16 2010 FINAL

http://slidepdf.com/reader/full/tntp-hisd-human-capital-analysis-summary-memo-8-16-2010-final 1/24

Overview

In December 2009, The New Teacher Project (TNTP) launched a partnership with theHouston Independent School District (HISD) to assist with the core initiative of the district’s

Strategic Direction to ensure that there is an effective teacher in every classroom. DuringPhase I of its engagement with HISD, TNTP conducted an extensive analysis of currenthuman capital policies and practices in the district in order to guide the design and

implementation of the core initiative. This memorandum provides a final summary of TNTP’sanalysis, which yielded four primary findings:

1. In order to attract top teaching talent, HISD must hire earlier, strategicallyrecruit from candidate pipelines that produce the most effective teachers,and utilize rigorous selection processes to identify candidates with the bestlikelihood of being effective. Page 3

2. The district’s appraisal systems do not accurately measure teachers’ abilityto promote student achievement, preventing HISD from using instructionaleffectiveness to inform key human capital decisions.

Page 6

3. Teachers do not receive professional development that helps them improvetheir performance in the classroom.

2

4. HISD does not maximize opportunities to retain and leverage effectiveteachers or to address teachers who demonstrate poor instructionalperformance. Page 14

TNTP shared the results of this analysis with the HISD Board of Trustees in two separatereports on April 29 and June 3, 2010. TNTP is engaged in the process of presenting itsanalysis to other stakeholders and has conducted briefings with teachers, HISD centraladministration staff, and community members. As TNTP enters the planning andimplementation phases of its work with HISD, it will continue to pursue and receive feedback

from stakeholders through targeted focus groups, community forums, and educator workingcommittees.

The New Teacher Project 1 August 2010

A U G U S T 2 0 1 0

Findings from Analysis of HumanCapital Policies and Practices inHouston Independent School District



Finding 1: In order to attract top teaching talent, HISDmust hire earlier, strategically recruit from candidatepipelines that produce the most effective teachers, and

utilize rigorous selection processes to identify candidateswith the best likelihood of being effective.

TNTP’s surveys of applicants to HISD between 2007 and 2009 identified a number of weaknesses in the district’s recruitment and hiring processes that have had the cumulativeeffect of causing the district to lose desirable teacher candidates. With more efficientprocesses and more strategic recruitment and selection, HISD will be able to optimize itssupply of new teachers to better meet the needs of schools and improve studentachievement.

It should be noted that HISD has significantly altered its recruitment practices for the 2010hiring season, and data from that year was not captured in our survey. While it is expectedthat many of the issues faced by candidates in prior years have been improved upon, HISDmust continue to use student outcome data to evaluate whether the changes in itsrecruitment and hiring practices have had the result of attracting new teachers who areultimately more effective in the classroom.

Summary of Data:i. In prior hiring seasons, HISD has lost desirable candidates due to flawed

communication processes and a late hiring timeline.ii. HISD should recruit strategically from the candidate pipelines that produce the most

effective teachers and pair those efforts with selection processes that candemonstrably identify candidates likely to be effective in the classroom.

i. In prior hiring seasons, HISD has lost desirable candidates due toflawed communication processes and a late hiring timeline.

Over the last three hiring seasons, teacher applicants to HISD have faced a lack of promptand clear communication. Respondents to TNTP’s applicant survey felt confused anddevalued by the application process, often due to unresponsiveness from the district and alack of clarity about how the hiring process would proceed.

Figure 1.1: Unhired Teacher Applicant Survey Responses Regarding HiringProcess

Teacher Response Percentage Responses

Unhired teacher applicants who indicate that they never received aresponse from HISD

62% 1,519

Of those who did receive a response, applicants who say thatcommunication from HISD about the hiring process was timely3 32% 571

Applicants who agree or strongly agree that communication from HISDabout the hiring process was clear

28% 564

3 Percentages indicate applicants who responded “agree” or “strongly agree.”




These delays and flawed communication processes caused the district to lose teachercandidates. The uncertainty created by delayed and unclear communications from HISDforced applicants to withdraw their applications or decline job offers from HISD to pursuetimelier job opportunities. Of the reasons cited for withdrawing an application or declining a

job offer, the length of the hiring process and the candidate receiving another job offer werethe most common.

Figure 1.2: Teacher Applicant Responses Regarding Reason for WithdrawingApplication or Declining a Job Offer

Teacher Response Percentage

I received another job offer. 70%

The application and hiring process took too long. 44%

The salary and benefits in HISD were insufficient. 17%

I was dissatisfied with the customer service of the HISD Human Resourcesdepartment.

17%

I had a negative interaction with a school principal or other district representative. 14%The hiring timeline was “important” or “very important” in my decision to withdraw myapplication or decline a job offer.

56%

It is important to note thatcandidates who withdrew theirapplications or declined a job offerincluded desirable prospects withcharacteristics indicating thepotential to promote studentachievement. The averageundergraduate GPA of candidateswithdrawing their application or

declining a job offer was 3.37, asopposed to the 3.17 average GPA of survey respondents who did not receive or accept anoffer from HISD.4 Though the number of responses from withdrawing or declining candidateswas relatively low, that group also appears more likely to be licensed in high-need areas,such as those listed in the table below.

Figure 1.3: Applicants Withdrawing an Application or Declining a Job OfferCertified in High-Need Licensure Areas

Licensure Area% of Declining

Applicants in EachLicensure Area (n=29)

% of WithdrawingApplicants in Each

Licensure Area (n=24)

% of Overall ApplicantPopulation in Each

Licensure Area(n=461)

Bilingual – Spanish 14% 17% 10%ESL – Generalist 35% 29% 23%

4 Survey of HISD teacher applicants. For research linking undergraduate GPA to applicant quality, see Andrew J.

Wayne and Peter Youngs, “Teacher Characteristics and Student Achievement Gains: A Review,” Review of Educational Research, 73 (1) (2003), 89-122; and Cathy W. Hall, Kris M. Smith, and Rosina Chia, “RelationshipBetween Metacognition and Affective Variables in College Achievement,” National Social Science Journal 19(1),(2002), 43-50.


“I wanted to work for HISD but it took HRforever to get back with me about even thesimplest answers.”

– Secondary English Language Arts Applicant

“I never heard back from schools I appliedto…when I finally did hear back from one, Ihad already been offered a position…[at]

YES Prep.”– Middle School Math/Science Applicant



Special Education 10% 25% 20%Science 14% 0% 7%i. HISD should recruit strategically from the candidate pipelines that

produce the most effective teachers and pair those efforts withselection processes that can demonstrably identify candidates likely tobe effective in the classroom.

TNTP’s analysis indicated that while some preparation programs are better than others atproducing high percentages of effective teachers, all pipelines produce both high and lowperformers. Analysis linking teacher effectiveness to degree-granting institutions produces acomplex picture with regard to the number of both high and low performing teachers;however, some conclusions can be drawn from the aggregate data. For example, HISD hasalmost a one in five chance of receiving a low performer from Sam Houston State, but lessthan a one in ten chance of receiving a low performer from Texas A&M - College Station. Thelatter program also produced a higher percentage of high performers.

Figure 1.4: Most Common Degree-Granting Institutions by Number of HISDTeachers and Percent of Teachers in Each EVAAS Performance Level, 2009-105

Going forward, HISD will need to refine this data by grade, license area, and effectivenesswith sub-groups of students (e.g., English Language Learners or students multiple yearsbelow grade level) in order to get richer data with which to inform recruitment strategies. Inaddition, these efforts must be paired with rigorous selection and evaluation processes toensure the district only hires candidates that have the potential to be effective in the

classroom. For any new selection models that are adopted, HISD should analyze the

5 HISD Human Resources and EVAAS data. Bolded numbers on the left side of the chart indicate number of totalteachers with a degree from the listed institution. Figures in parentheses are the total number of teachers withdegrees from the listed institution who have multi-year value-added data. High-performing teachers are defined asbeing in the top decile of performers in at least one subject and not in the bottom quartile of performers in anyother subject when using two or three year average EVAAS percentile ranks. Low-performing teachers are definedas being in the bottom decile in at least one subject and not in the top quartile in any other subject.




demonstrated student achievement growth of the teachers hired through those processes toensure the models can consistently identify effective candidates.




Finding 2: The district’s appraisal systems do not

accurately measure teachers’ ability to promote student

achievement, preventing HISD from using instructional

effectiveness to inform key human capital decisions.

A wide body of research has established that teacher effectiveness is the single mostimportant school-based factor impacting student achievement. In order to maximize studentachievement, HISD must base key human capital decisions on accurate data regardingteacher performance levels. These decisions (including compensation, retention, dismissal,the granting of term contracts, and professional development) are currently made in avacuum of information, as teacher appraisal and survey data demonstrate that the districtcurrently has no reliable data differentiating teachers’ instructional performance. HISD mustdevelop new, comprehensive systems to appraise teacher performance that includemeasures of a teacher’s impact on student achievement as a predominant factor.

Summary of Data:i. Neither teachers nor principals believe the district’s current teacher appraisal

practices accurately assess teachers’ instructional performance.ii. By rating almost all teachers as “Proficient” or “High Performing” regardless of their

ability to promote student achievement, the teacher appraisal process fails toprovide a basis for continuous professional improvement.

iii. The Staff Review Process conducted in March 2010 yielded a greater degree of differentiation among teacher performance levels, pointing to potential opportunitiesfor the improvement of the teacher appraisal system.

iv. Teachers want to be assessed based on multiple factors including studentachievement growth, but have concerns about the district’s current value-addedmeasures.

i. Neither teachers nor principals believe the district’s current teacherappraisal practices accurately assess teachers’ instructionalperformance.

The current teacher appraisal systems in HISD, the Professional Development and AppraisalSystem (PDAS) and the Modified Professional Development and Appraisal System (MPDAS),evaluate teachers’ performance in eight domains, each of which is assessed on a four pointscale that ranges from “High-Performing” to “Unsatisfactory.” Teachers are appraised onceeach year. Term or continuing contract teachers who are rated “Proficient” or higher in eachdomain can choose to be appraised using MPDAS during the following year. Under MPDAS, infive of the eight appraisal domains, teachers accept the previous year’s ratings as thehighest possible ratings they can receive that year, and appraisers gather data for those five

domains using walk-through observations rather than formal observations.




According to teachers and principals, PDAS and MPDAS do not accurately differentiateteacher effectiveness, hindering HISD’s ability to assess instructional performance,meaningfully inform professional development, and address instances of poor performance.

Figure 2.1: Teacher and Principal Opinions Regarding the PDAS/MPDAS Process6

Percentage of Teachers Who “Agree” or “StronglyAgree” That the PDAS/MPDAS Process… Teachers(PDAS) Teachers(MPDAS) Principals(PDAS/MPDAS)… allows appraisers to accurately assess teachers'instructional performance. (i.e., the appraisal ratings reflectteacher effectiveness in promoting student achievement).

44% 55% 28%

…recognizes effective instructional performance. 48% 56% 29%

...identifies and offers concrete steps to remedy poor performance.

39% 46% 19%

...helps teachers improve their instructional performance. 43% 50% 20%

Total respondents 5,108 3,183 114

ii. By rating almost all teachers as “Proficient” or “High Performing,”regardless of their ability to promote student achievement, the teacherappraisal process fails to provide a basis for continuous professionalimprovement.

One likely reason for principals’ and teachers’ lack of confidence in the current appraisalsystem is that its results do not match the reality of the instructional performance that theyobserve in their schools daily. Survey responses indicate that principals and teachersrecognize a significant percentage of low-performing and developing teachers in theirbuildings, yet almost no teachers are rated “Below Expectations” or “Unsatisfactory” onPDAS or MPDAS.

Figure 2.2: Average Perceptions of Teacher Performance Held by Principals andTeachers Compared with Actual Teacher Performance on PDAS and MPDAS

between 2006-07 and 2008-097

An analysis of teacher appraisal results as compared with school-level student outcomesindicates some differentiation in the percentage of teachers who receive the top ratingversus the second highest, however even at low-performing schools, very few teachersreceive ratings of “Below Expectations” or “Unsatisfactory.” This communicates to teachersthat all performance is at or above the desired standard and contributes to a persistent

6 Surveys of HISD teachers. Percentages indicate respondents who selected “Strongly agree” or “Agree.”

7 Surveys of HISD teachers and principals and HISD Human Resources records. Survey respondents were asked to

identify the percentage of teachers at their school performing in each of the four groups. These percentages werecalculated by averaging all of the respondents’ distributions. Principal n=81. Teacher n=2,871. The PDAS ratingcurve indicates the percentage of teacher appraisals that had all domains scored in the indicated rating category orhigher over the last three years. n=32,345.




culture of lowered expectations for teacher performance and the flawed assumption thatmost teachers do not have any room for improvement in their instructional practices.

Figure 2.3: PDAS/MPDAS Results for Teacher by School TEA Rating for the 2008-2009 School Year8

Texas EducationAgency Rating of Schools

All DomainsRated "ExceedsExpectations"

All Domains

Rated"Proficient" Or

Higher

All Domains

Rated "BelowExpectations" Or

Higher

At Least One DomainRated“Unsatisfactory”

Exemplary 41% 57% 2% 0.0%

Recognized 32% 66% 2% 0.1%

Acceptable 23% 73% 4% 0.0%

Unacceptable 8% 85% 7% 0.3%

All Teachers 30% 67% 3% 0.1%

Figure 2.4: 2008-09 PDAS/MPDAS Results by Teacher from Schools Rated“Academically Unacceptable” by the Texas Education Agency in 2008-20099

Schools Rated"AcademicallyUnacceptable"

All Domains Rated"Proficient" Or

Higher

All Domains Rated"Below

Expectations" Or Higher

At Least OneDomain Rated

“Unsatisfactory”Total Teachers

Attucks Middle School 40 0 0 40

Dowling Middle School 85 4 1 90

Fondren Middle School 42 1 0 43

Jones High School 43 2 1 46

Kashmere High School 39 3 0 42

Lee High School 92 7 0 99Ryan Middle School 17 1 0 18

Sharpstown High School 67 8 0 75

Sterling High School 59 3 0 62

Westbury High School 93 8 0 101

Worthing High School 38 7 0 45

8 HISD Human Resources Records.





iii. The Staff Review Process conducted in March 2010 yielded a greaterdegree of differentiation among teacher performance levels, pointingto potential opportunities for improvement of the teacher appraisalsystem.

During the 2009-2010 school year, HISD implemented a Staff Review Process as aperformance management measure. In this process, principals categorized each of theteachers at their schools according to their assessment of the teacher’s level of effectiveness. Principals met with members of HISD’s administration to discuss the evidencefor these categorizations and the actions that principals were taking to retain highperformers, provide struggling teachers with professional development, and pursuenonrenewal or the extension of three-year probationary contracts to a fourth year forunderperforming probationary teachers..

In the Staff Review Process, principals differentiated the performance of their teachers to agreater degree than through the PDAS/MPDAS process, and were much more likely to rateteachers in one of the two lowest categories. Differences between the implementation of the

Staff Review Process and PDAS/MPDAS shed light on needed reforms of teacher appraisal.First, the review of teacher ratings by HISD administrators and the requirement thatprincipals provide evidence for performance categorizations discouraged principals frominflating their decisions. Second, in cases where teacher-level EVAAS data was available, theinformation was used by principals and HISD administrators to inform performancecategorization decisions. Finally, principals were held accountable to act on thecategorizations they assigned their teachers (e.g., developing professional growth plans forstruggling teachers and working with HISD Professional Development Services to plantraining resources) so that the Staff Review Process was meaningfully connected to theprincipal’s primary role as the instructional leader of his/her school. These three elements –norming and oversight of appraisal ratings; incorporation of student outcome data; andmeaningful connections to key decisions – must be part of any redesign of the appraisalsystem.

Figure 2.5: Rating Category Distribution from PDAS/MPDAS for 2008-2009 School Year and Staff Review Process for 2009-10 School Year10

HighlyEffective/Exceeds

Expectations

Proficient DevelopingLow-

Performing/Unsatisfactory

TotalTeacher s Rated

2009-10 Staff Review Process 28% 45% 21% 6% 11,249

2008-09 PDAS/MPDAS 30% 67% 3% 0% 10,572

Figure 2.6: Staff Review Process Results by Texas Education Agency School Ratingfrom 2008-09

TEA SchoolRating

Highly Effective Proficient Developing Low-PerformingTotal

Schools

Exemplary 38% 42% 17% 3% 102

Recognized 28% 47% 20% 5% 68





Acceptable 21% 44% 26% 9% 41

Unacceptable 12% 54% 26% 8% 11

Total Teachers 2,909 4,787 2,190 587

However, even in the context of the Staff Review Process, rating inflation and a culture of low expectations persist. While teachers with multiple years of strong value-added results

were much more likely to be rated as “Highly Effective” or “Proficient” during the Staff Review Process a significant number of low performers continued to receive a satisfactorycategorization. Of those teachers identified as low performers by their multi-year EVAASdata, 36 percent received a Staff Review Process categorization of “Proficient” or higher.

Figure 2.7: 2009-10 Staff Review Process Categorizations by Multi-Year EVAASPerformance Level11

Staff ReviewCategorization

Percent of EVAAS HighPerformers

Percent of EVAASMiddle Performers

Percent of EVAASLow Performers

Highly Effective 78% 34% 8%

Proficient 20% 44% 28%

Developing 2% 16% 42%

Low-Performing 0% 5% 22%

Total Teachers 308 1396 275

iv. Teachers want to be assessed based on multiple factors includingstudent achievement growth, but have concerns about the district’scurrent value-added measures.

Since the promotion of student learning is a teacher’s foremost responsibility, an effectiveteacher appraisal system must include measures of student growth as the predominantfactor. HISD has collected EVAAS data using the results of the Texas Assessment of

Knowledge and Skills (TAKS) and the Stanford Achievement Test for the last four years, andis technically well-positioned to utilize its current growth measures as a factor in teacherappraisal. HISD should combine current student growth measures with additionalassessments and processes to assess student learning gains in a wider range of subjectsand grades, and use these measures as the foundation of teacher appraisals. Survey datafrom teachers indicate that most agree multiple factors, including student growth, should beincluded in their appraisals.

Figure 2.8: Teachers Who Believe the Following Criteria Are “Very Important” Or“Important” In A Fair and Comprehensive Evaluation System12

Evaluation Criteria PercentTeacher mastery of content knowledge 96%

Student engagement in the classroom 94%Classroom management 95%Professionalism 90%

Instructional technique 89%

11 HISD Staff Review Process data and EVAAS multi-year averages.

12 Surveys of HISD teachers Percentage of respondents who selected “Very Important” or “Important.”




Measures of teacher impact on student learning 88%Evidence of preparation and planning 86%

Measures of student academic progress 83%Improvement since last evaluation 76%

Total Respondents 4,853

Teachers want appraisals to include their impact on student performance, and they supportthe use of tests to measure growth, as 87 percent of teachers believe that some form of test-based assessment should be used to measure their impact on student learning.However, teachers lack confidence in EVAAS, as only 18 percent of teachers surveyed inMarch and April 2010 believe that it provides a fair and accurate measure of a teacher’simpact on student learning. This uneasiness with value-added measures can be partiallyexplained by teachers’ lack of understanding about the measures themselves. After HISDconducted trainings of teachers on EVAAS methodology in June 2010, a significantpercentage of participants indicated that their confidence in value-added measures likeEVAAS had increased. After participating in trainings, 56 percent of teachers expressedbelief in the system’s fairness and accuracy.13

Figure 2.9: Teacher Responses to the Question: “I believe HISD’s measures of ateacher’s effect of students’ standardized test score growth, such as EVAAS,provide a fair and accurate measurement of a teacher’s impact on student

learning”

Furthermore, teachers indicate support for the use of school- and teacher-generatedassessments in their appraisals in addition to TAKS or other state standardized exams. Thissuggests that in addition to using measures of standardized test score growth, teachers wishto have their impact on students be assessed using methods that they consider to beauthentic and connected to their daily practice.

Figure 2.10: The Percentage of Teachers Who Believe That These IndividualComponents Should Be Used As Part of the Measure of Their Impact on Student

Learning 14

Measure of Student Learning PercentStudent standardized test scores from assessments such as TAKS or standardized end of course exams

37%

Student standardized test score growth that takes into account where studentsbegin the year, such as EVAAS

48%

Student academic growth based on school- or teacher-generated assessments 72%Student work products assessed according to district-wide, standardized rubrics 48%Student graduation rates 23%Student engagement levels 52%School-wide student achievement 43%Surveys of students in secondary grades 21%Surveys of parents 27%Teacher self-assessments 51%

Total Respondents 4,663

13 It should be noted that teachers participating in trainings responded with higher levels of confidence in EVAASprior to trainings than the average teacher responding to the March and April teacher surveys.

14 Surveys of HISD teachers and HISD exit survey of teachers participating in Foundation Level EVAAS Trainingsheld between June 8 and June 28. Teacher survey n=4,505. Exit survey n=636. Data accurate as of July 28, 2010.




Finding 3: Teachers do not receive professionaldevelopment that helps them improve their performancein the classroom.

Just as teacher appraisal systems must reflect the differences in the performance levels of

individual teachers, professional development must be responsive to individual growthneeds and cannot be conducted using a “one size fits all” approach. TNTP’s analysisindicates that current professional development efforts, both central and at the school-level,have not been effective at improving teacher practice or student learning. As HISDrestructures its performance management systems, it must consider new ways of aligningthe identification of teacher development needs with individually targeted, job-embeddedprofessional development opportunities.

Summary of Data:i. Teachers do not currently receive professional development that addresses their

individual growth needs or helps them improve their ability to promote studentachievement.

ii. Principals do not have the time and training necessary to appraise and develop

teachers’ instructional performance.

i. Teachers do not currently receive professional development thataddresses their individual growth needs or helps them improve theirability to promote student achievement.

Among the many repercussions of an ineffective teacher appraisal system, one of the mostdetrimental is the inability to provide teachers with targeted professional development toimprove their instructional performance. HISD teachers reported that current professionaldevelopment offerings do not address their individual needs or help them improve theirability to promote student achievement.

Figure 3.1: Percentage of Teachers Who “Agree” or “Strongly Agree” to theFollowing Statements: The professional development I receive from HISD... 15

Teacher ResponsePercentage Respondents

...adequately addresses my individual needs as an educator. 51% 5,414

...is tailored according to the needs of my students as determined byclassroom-based or standardized assessments.

45% 5,390

...is tailored according to feedback and/or development domains from myPDAS/MPDAS.

39% 5,379

...helps me to improve my effectiveness in promoting studentachievement.

52% 5,369

15 Surveys of HISD teachers. Percentages indicate respondents who selected “Strongly agree” or “Agree.”




Teachers and principals believe that individualized supports, such as observing otherteachers and participating in one-on-one coaching, are more effective in improving teachers’instructional performance than school-wide programs. In making decisions aboutprofessional development offerings, HISD should prioritize the offerings that are mostresponsive to individual needs as determined by a comprehensive appraisal system and thatcan be shown to have demonstrable positive effects on teacher performance.

Figure 3.2: “Effective” or “Highly Effective” Professional Development Methods16

Professional Development Support Teachers PrincipalsSchool-wide PD generated by school staff 53% 60%

School-wide PD generated by central HISD staff 47% 31%School-wide PD generated by external vendors 46% 38%

Observing other teachers 68% 79%Grade/subject level learning communities 70% 80%

One-on-one coaching 66% 82%

ii. Principals do not have the time and training necessary to appraise anddevelop teachers’ instructional performance.

TNTP’s surveys of principals indicate that one of the major barriers to effective teacherappraisal and development at the school-level is the lack of time and training. As HISDredesigns its teacher performance management processes, it is clear that significantalterations to principal time allocation and training must be considered in order for teacherprofessional development and appraisal to be conducted successfully.

• Only 18 percent of principals feel that they have sufficient time to focus on teacherappraisals and development.

• Only 32 percent of principals feel that they are given adequate training and supportfrom HISD on how to appraise and develop teachers’ instructional performance.

Figure 3.3: Principals’ Reported Average Time Allocation17

Actual TimeSpent

Principal ActivityDesired

Time Spent17% Visiting classrooms and observing teachers 32%7% Providing teachers with feedback on instruction 14%6% Planning and delivering professional development 7%

16% District required paperwork 5%8% Reviewing student achievement data 9%9% Attending staff or district meetings 3%57 Total Respondents 55

16 Surveys of HISD teachers. Percentage of respondents who selected “Highly Effective” or “Effective.”

17 Surveys of HISD principals. Average “Actual’ and “Desired” time allocation taken from principals who indicated

that their time is not distributed in a way that best supports student growth.




10th and below 96%

The responses to TNTP’s survey of HISD teachers indicate that the district’s highestperforming teachers, those who have demonstrated high levels of student growth acrossmultiple years, do not consistently plan to remain in the district longer than low performers.

Figure 4.2: Teachers Planning to Leave HISD within Three Years in the Top andBottom Deciles of EVAAS Scores19

Subject EVAAS PercentilePercent of Teachers Who Plan

to Leave HISD Within Three Years

Reading (n=99) 90th and above 19%

10th and below 24%

Math (n=97) 90th and above 23%

10th and below 13%

Language (n=91) 90th and above 11%

10th and below 23%

19 Surveys of HISD teachers.


Comments from Teachers Planning to Leave HISD in the Next Three Years

“There are very few opportunities for monetary/professional advancement foreffective teachers. Effective teachers are not being identified, and ineffectiveteachers continue to make the load heavier for the whole school.”

– Elementary Bilingual Teacher

“Teacher leadership opportunities are limited. Although many of my colleaguesand I have come into teaching from other careers (lawyers, consultants, engineers)we are seen as having a very small role and skill-set…Regarding professionaldevelopment, if I learn and master the limited high quality professionaldevelopment I occasionally receive, I am sentenced to be bored over the nextseveral years as I am forced to hear the same thing over and over. We differentiatefor student learners, but not for teachers…I wish the district wouldn't just focus onhow to get rid of the few bad teachers, but also on how to keep the good ones. ”

– Secondary Reading Teacher with Three-Year EVAAS Performance in the TopQuartile



i. HISD teachers do not feel confident that ASPIRE awards are reflectiveof their classroom performance, likely due in part to the use of fluctuating single-year EVAAS results.

Beginning in 2006, HISD launched the ASPIRE Award Program and has distributed rewards toteachers, administrators, and other school-based staff using student achievement outcomes,including value-added measures of teacher impact on student learning. The ASPIRE programhas had a measurable impact on the retention of effective teachers20, but since the retentionrate of teachers receiving the highest amount of individual financial reward is only slightlyhigher than that of teachers receiving no individual reward at all, it is clear that HISD has notmaximized the effect of its incentive program. HISD should take additional steps to refine itsteacher compensation structure in a way that allows it to reward effective teaching andencourage high-performing teachers to remain in HISD.

Figure 4.3 2008-09 Teacher Retention Rate and Amount of 2008 Strand 2Individual ASPIRE Award21

Amount of Strand 2 Individual Award Retention Rate

$0 92.3%$1 - $625 92.7%

$626 - $1875 93.1%$1876 or more 93.8%

Total Teachers Considered10,370

HISD has conducted annual surveys of teachers on their opinions regarding the ASPIREAward Program. Those surveys indicate that while most teachers are not opposed to theconcept of performance pay, the limited impact of the ASPIRE program on the retention of highly effective teachers can be partly explained by the fact that many teachers areconfused about the program’s implementation and unsure if it is executed accurately andfairly.

Figure 4.4: HISD Surveys Regarding ASPIRE Awards 2006-07 through 2008-0922

Teacher Response2006-2007

2007-2008

2008-2009

Respondents “in favor” or “somewhat in favor” of the concept of teacher performance pay

57% 64% 55%

Respondents “in favor” or “somewhat in favor” of the ASPIRE rewardprogram

45% 53% 47%

20 SAS Institute, “Analysis of a Teacher Pay-For-Performance Program: Determining the Treatment Effect and

Overall Impact.”

21 HISD data on ASPIRE individual awards, Human Resources separation data, and a survey of current HISDteachers.

22 HISD Surveys on ASPIRE Program conducted annually beginning in 2008. Some questions were not asked inevery year of the survey. The 2008-2009 survey was distributed soon after the Board of Education approved theuse of value-added data in teacher appraisals. The uncertainty felt by many teachers as a result of this policychange is likely reflected in their survey responses.




Respondents who stated “absolutely” or “mostly” that there is a connectionbetween classroom instruction and performance pay results

40% 45% 38%

Respondents who state that they understood "most aspects" of theperformance pay model paid out that year or that they understood it"completely"

27% 39% 17%

Respondents who rate their level of understanding of how the ASPIRE

Awards were calculated as “high” or “very high”

22% 27% 22%

Respondents who indicate that they have received training about ASPIRE 85% 79% 71%

Respondents who believe that the maximum ASPIRE award amountadequately recognized their efforts to increase student progress

N/A 32% 29%

In part, these concerns may stem from the instability of the single year value-added scoresthat are used to calculate ASPIRE Awards. Over three years, significant percentages of teachers shift from the top and bottom quintile of EVAAS scores in each subject, and themajority of teachers have had both positive and negative value-added scores. Fluctuation insingle-year scores may contribute to a lack of credibility among teachers regarding thereward system’s ability to reward excellence accurately. Without teachers’ trust in thedistrict’s value-added measures, ASPIRE cannot effectively incentivize retention andeffective instructional practice. Therefore, HISD should consider using multi-year value-added scores to inform financial incentives and other means of retaining effective teachers.

Figure 4.5: Teachers with Fluctuating EVAAS Scores across Three Years of Data23

EVAAS Subject

Teacher Appeared in Top

and BottomQuintile

Teacher Appearedin Either 1st and4th or 2nd and 5th

Quintile

Teacher HadPositive andNegative CGI

Scores

Total Teacherswith Three Years of

EVAAS DataLanguage 21% 39% 73% 694Math 16% 34% 70% 549Reading 18% 37% 71% 726Science 19% 33% 75% 192Social Studies 17% 36% 73% 240

ii. Many teachers support a restructuring of compensation that wouldmake advancement on the salary schedule reflective of studentlearning growth, which could be accomplished by diverting resourcesfrom salary increases based on factors unrelated to teachereffectiveness in HISD.

In addition to financial bonuses, many teachers support a more significant restructuring of the compensation system that would allow for differences in baseline pay according toclassroom performance. The majority of teachers stated they would agree to a differentiatedsalary schedule based on student learning growth. A change such as this would be adeparture from the current compensation system, which includes a traditional salaryschedule along with performance-based bonuses.

Figure 4.6: Agreement with: “To what extent do you agree that the followingfactors would be an appropriate part of a compensation system that fairly

rewards teachers for instructional excellence?”24





Teacher Response Percent

Accelerated salary schedule advancement for teachers who consistentlyshow high levels of student academic growth

61%

Opportunities for school leadership positions for teachers who show highlevels of student academic growth

65%

Financial rewards for all teachers in a grade or subject based on the

academic growth of students in that grade or subject

62%

Financial rewards for all teachers in a school based on the academicgrowth of students in that school

66%

Financial rewards based on annual appraisal results 47%Increased baseline salaries and salary step increases for all teachers 78%

Total Respondents4,615

The current teacher compensation structure in HISD and most districts around the country,rewards teachers based on seniority and advanced degrees rather than on classroomperformance. TNTP’s analysis of teacher performance based on value-added measures inHISD indicates that neither seniority nor advanced degrees are reliable predictors of ateacher’s effectiveness. In some subjects, like Language and Reading, high performers tendto be more experienced than low performers, but the opposite is true in subjects like Mathand Science. Similarly, across subjects, having an advanced degree does not appear to have

an impact on a teacher’s effectiveness in the classroom. This data indicates that whileteacher experience and advanced degrees can be valuable assets in the classroom, HISDshould implement financial incentives that directly encourage the promotion of studentachievement growth rather than indirect teacher inputs.

Figure 4.7: The Average Number of Years Taught by a Teacher in HISD by EVAASPercentile and Subject Area (n=2075)25

EVAAS SubjectEVAAS 90th

Percentile or AboveEVAAS 11th to 89th

PercentileEVAAS 10th Percentile

or Below

Language (n=1760) 15.9 14.4 11.4

Math (n=1645) 10.5 13.1 13.5

Reading (n=1736) 14.2 14.0 13.5

Science (n=1229) 12.2 12.6 15.7

Social Studies(n=1331)

12.3 13.5 12.7

Figure 4.8: Percent of Teachers with an Advanced Degree by EVAAS Percentileand Subject Area 26

EVAAS SubjectEVAAS 90th

Percentile or AboveEVAAS 11th to 89th

PercentileEVAAS 10th

Percentile or Below

Language (n=1760) 28% 30% 30%

Math (n=1645) 25% 29% 32%

Reading (n=1736) 31% 30% 34%

24 Surveys of HISD teachers. Percentages indicate teachers who responded “Strongly Agree” or “Agree.”






Science (n=1230) 25% 31% 33%Social Studies(n=1332) 27% 30% 32%

A restructuring of the teacher compensation system that would incentivize excellentperformance and retain highly effective teachers could reallocate financial resources away

from salary schedule increases based solely on seniority and advanced degrees and towardssalary increases based on demonstrated classroom performance.




iii. HISD has failed to exit consistently ineffective teachers.

HISD’s inability to retain high-performers at a significantly higher rate than low-performershas been compounded by a failure on the part of the district to capitalize on opportunities to

exit consistently ineffective teachers. Very few teachers are exited from the district due totheir performance in the classroom, including those whom principals indicate were poorperformers. This includes probationary teachers whose contracts make nonrenewal forperformance reasons much easier than the nonrenewal of teachers who have received termcontracts after completing their third year of teaching.

• 0.6 percent of all probationary teachers between 2005-2006 and 2008-2009 werenonrenewed for performance reasons.

• 56 percent of principals say that at least one poorly performing probationary teacherat their school received a term contract in the last five years.

• In the last five years, 50 percent of principals have not even attempted to nonrenewor terminate a poorly performing term or continuing contract teacher.

As in other school districts across the country, the number of continuing or term contractteachers who have been nonrenewed or terminated for performance reasons is alsoextraordinarily low. To address this problem, HISD must couple more effective means of identifying poor performance through the appraisal system with more efficient methods of exiting teachers whose performance does not improve after receiving targeted support.

Figure 4.9: Percentage of Continuing or Term Contract Teachers Terminated orNonrenewed for Performance by School Year27

Year Teachers Exited

from HISD for Performance

Percentage Exited for Performance

Total Continuingor Term Contract

Teachers

2005-2006 7 0.1% 8,482

2006-2007 12 0.1% 8,504

2007-2008 13 0.2% 8,4592008-2009 18 0.2% 8,562

27 Termination and nonrenewal data provided by HISD Professional Standards Office.




Observers may question the utility of exiting underperforming teachers for fear that theteachers replacing them may not be significantly more effective. However, when comparingthe performance of HISD’s lowest performing teachers to the performance of average firstyear teachers, an analysis of value-added data indicates there are groups of teachers thatare demonstrably less effective than new teachers. Replacing HISD’s least effective teacherswould therefore yield immediate benefits to student performance.

Figure 4.10: Distribution of Teachers’ Mathematics 2- and 3-Year Average Value-Added Scores28

28 HISD Human Resources Records. Average performance of lowest 10% of teachers is based on two and threeyear value-added data. Average performance of first year teachers is based on single year data from SY 2008-09.




Conclusion

In response to data such as the findings presented above, HISD has made thetransformation of human capital management a top priority and included “An Effective

Teacher in Every Classroom” as one of the five core initiatives at the heart of its StrategicDirection. This Core Initiative includes four key strategies to address the challengesdescribed above:

• Strengthen recruiting and staffing policies and practices to attract top talent.

• Establish a rigorous and fair teacher appraisal system to inform key decisions.

• Provide effective individualized support and professional development for teachers.

• Offer meaningful career pathways and differentiated compensation to retain and

leverage the most-effective teachers.

TNTP has been working with HISD leadership to develop a five-year master plan to transform

the district’s human capital systems and lead to HISD having an effective teacher in every

classroom. This plan includes the development of new tools and processes, coordination and

monitoring of school-level implementation activities, and building stakeholder buy-in forreform. Throughout the five-year initiative, data on implementation outcomes will

continuously be monitored and real-time adjustments will be made to strategies and tactics,

as required. Ambitious, concrete goals have been set for the success of the Effective

Teachers Core Initiative by the 2014-15 school year, including:

• Fewer than 10 percent of new hires are ineffective in their first year in HISD asidentified through the Staff Review Process and annual teacher appraisal instrument,and at least 50 percent of new hires are effective or better by their third year in thedistrict.

• The distribution of teacher appraisal ratings at the district and school-level is alignedwith student performance outcomes and with survey data indicating teachers’ andprincipals’ assessments of teacher performance.

• 100 percent of probationary teachers who receive a term contract have performed atthe “Effective” level or above, and at least 90 percent of teachers rated “Ineffective”either improve their performance to a level above “Ineffective” or are removed fromteaching in HISD within one year of receiving the “Ineffective” rating.

• In surveys, at least 85 percent of teachers and 90 percent of principals agree orstrongly agree that teacher appraisal in HISD is a fair and accurate measure of teacher performance.

• Annual appraisal data shows a significant increase in the overall effectiveness of HISD’s teaching force, and a significant percentage of teachers improve theirperformance at least one rating level from year to year.

• At least 85 percent of teachers and at least 90 percent of principals agree or strongly

agree that HISD’s teacher support and development systems and processes addressteachers’ individual needs and help them improve their performance.

• HISD retains highly effective teachers at a rate at least twice the rate of teachersrated “below effective” as measured through the Staff Review Process and teacher-level value-added data, when available.

• At least 85 percent of teachers and at least 90 percent of principals agree or stronglyagree that HISD is doing enough to leverage and retain its most effective teachers.




Further information regarding HISD’s core initiative of “An Effective Teacher in Every

Classroom” can be obtained by emailing [email protected].


tntp hisd human capital analysis summary memo 8 16 2010 final

Documents