teach for america study 2015

8/20/2019 Teach for America Study 2015

1/37

EA15-536-2

Evaluation of Teach For America:

2014-2015

Department of Evaluation and Assessment

Mike MilesSuperintendent of Schools


2/37

Evaluation of Teach For America: 2014-2015

ii

This page is intentionally left blank.


3/37


iii

Approved Report of the

Department of Evaluation and Assessment

EA15-536-2

January 2015

This report was originally prepared for Commit! and the Texas Instruments Foundation. The report has beenupdated to add abstract and summary/recommendation sections.

Nolan Rett Mickelson

Amber McEnturff

Evaluation of Teach For America:

2014-2015

Joan Bush

Manager – Program Evaluation

Nancy Kihneman, Ph.D.

Director – Program Evaluation

Cecilia Oakeley, Ph.D.

Assistant Superintendent – Evaluation and Assessment

Mike MilesSuperintendent of Schools


4/37


iv

This page is intentionally left blank.


5/37


v

Table of Contents

ABSTRACT ................................................................................................................................................... 1

PURPOSE AND SCOPE OF THE EVALUATION ........................................................................................ 1

MAJOR EVALUATION QUESTIONS AND RESULTS ................................................................................. 2

How many first-year teachers were in the Dallas ISD each year, and how many of these were TFA

teachers? ................................................................................................................................................... 2

Methodology .......................................................................................................................................... 2

How does classroom effectiveness of TFA teachers compare with other first-year teachers in the

district? ...................................................................................................................................................... 3

Methodology .......................................................................................................................................... 3

Results ................................................................................................................................................... 4

How does the STAAR performance of students of TFA teachers compare with students of other

teachers in the district?.............................................................................................................................. 7

Methodology .......................................................................................................................................... 7

Results ................................................................................................................................................... 8

How does the retention of TFA teachers compare with non-TFA teachers in the district over time? ..... 25

Methodology ........................................................................................................................................ 25

Results ................................................................................................................................................. 25

SUMMARY AND RECOMMENDATIONS ................................................................................................... 28

Summary .............................................................................................................................................. 28

Recommendation ................................................................................................................................. 29


6/37


vi

List of Tables

Table Page

1 Number of First-Year Teachers and TFA teachers by School Year Cohort .............. 2

2 Number of Teachers with Valid CEI Scores by Cohort and Subject ......................... 4

3 Logistic Regression Final Model Results for STAAR Mathematics

Grades 4 to 5 .......................................................................................................... 12

4 Logistic Regression Final Model Results for STAAR Mathematics

Grades 6 to 8 .......................................................................................................... 13

5 Logistic Regression Final Model Results for STAAR Algebra I

End-of-Course Exam ............................................................................................... 13

6 Logistic Regression Final Model Results for STAAR Reading

Grades 4 to 5 .......................................................................................................... 17

7 Logistic Regression Final Model Results for STAAR Reading Grades 6 to 8 ......... 178 Logistic Regression Final Model Results for STAAR English I


9 Logistic Regression Final Model Results for STAAR English II


10 Logistic Regression Final Model Results for STAAR Science Grade 5 .................. 22

11 Logistic Regression Final Model Results for STAAR Science Grade 8 .................. 22

12 Logistic Regression Final Model Results for STAAR Biology

End-of-Course Exam ............................................................................................... 2313 Logistic Regression Final Model Results for STAAR Social Studies

Grade 8 ................................................................................................................... 25


7/37


vii

List of Figures

Figure Page

1 Average CEIs by Cohort and Content Area for TFA vs. Non-TFA Teachers ............ 5

2 Percentage That Met Expectations on Spring 2014 STAAR for Students of

First-Year TFA and Non-TFA Teachers .................................................................... 9


Second-Year TFA and Non-TFA Teachers……………………………………………. 9


Third- through Fifth-Year TFA and Non-TFA Teachers ……………………………… 9

5 Percentage That Met Expectations on STAAR Mathematics for Students of

TFA and Non-TFA Teachers (Grades 3 to 5) ………………………………………… 10

6 Percentage That Met Expectations on STAAR Mathematics for Students of

TFA and Non-TFA Teachers (Grades 6 to 8) .......................................................... 117 Percentage That Met Expectations on STAAR Algebra 1 EOC for Students of

TFA and Non-TFA Teachers……………………………………………………………. 11

8 Percentage That Met Expectations on STAAR Reading for Students of

TFA and Non-TFA Teachers (Grades 3 to 5) .......................................................... 14

9 Percentage That Met Expectations on STAAR Reading for Students of

TFA and Non-TFA Teachers (Grades 6 to 8) .......................................................... 15

10 Percentage That Met Expectations on STAAR English 1 EOC for Students

of TFA and Non-TFA Teachers………………………………..……………………….. 1511 Percentage That Met Expectations on STAAR English 2 EOC for Students of

TFA and Non-TFA Teachers ................................................................................... 16

12 Percentage That Met Expectations on STAAR Science for Students of

TFA and Non-TFA Teachers (Grade 5) ………………………………..……………... 20

13 Percentage That Met Expectations on STAAR Science for Students of

TFA and Non-TFA Teachers (Grade 8)……………………………………………….. 20

14 Percentage That Met Expectations on STAAR Biology EOC for Students of

TFA and Non-TFA Teachers……………………………………………………………. 2115 Percentage That Met Expectations on STAAR Social Studies for Students of

TFA and Non-TFA Teachers (Grade 8) .................................................................. 24

16 2009-10 Cohort Retention Rate by Year Began ..................................................... 26




8/37


viii




9/37


1

E

VALUATION OF

T

EACH

F

OR

A

MERICA

:

2014 2015

Project Evaluators: Nolan Rett MickelsonAmber McEnturff

ABSTRACT

Teach For America (TFA) is a program that recruits recent college graduates and prepares them

to teach for two years in public schools. Since the 2009-2010 school year, 563 TFA teachers have

worked in the Dallas Independent School District (ISD). Analyses of variance comparing the effectiveness

of each cohort of TFA teachers in their first and second years to other teachers in their first or second

years with the Dallas ISD indicated that most TFA cohorts were more effective, on average, than the

comparison groups for both years. Several differences were statistically significant in mathematics and

science. Using data from the 2013-14 school year in a series of logistic regressions, the evaluators

assessed the impact of various teacher and student characteristics on students’ mathematics, reading,

science, and social studies STAAR passing rates. For mathematics, having a TFA teacher was

associated with significantly higher odds of passing across all grade levels after adjusting for student

characteristics and teacher experience; results varied for other subjects and grade levels. TFA interns

have returned to teach in the Dallas ISD for a second year at very high rates. As expected, a greater

proportion of non-TFA than TFA teachers remained in district classrooms for three years or longer.

PURPOSE AND SCOPE OF THE EVALUATION

Teach For America (TFA) is a program that recruits high-achieving recent college graduates and

prepares them to teach for two years in urban and rural public schools. TFA teachers have worked in the

Dallas Independent School District (ISD) since the 2009-2010 school year. The purpose of this report is to

summarize the classroom effectiveness and retention of TFA teachers compared with other teachers in

the district and to review the performance of their students on the State of Texas Assessments of

Academic Readiness (STAAR). This report was originally produced for Commit! and the Texas

Instruments Foundation. It has been updated to include abstract and summary/recommendation sections.


10/37


2

MAJOR EVALUATION QUESTIONS AND RESULTS

How many first-year teachers were in the Dallas ISD each year and how many

of these were TFA teachers?

Methodology

The Dallas ISD’s Human Capital Management Department provided a list of TFA teachers with

their hire dates. The group of first-year non-TFA teachers included: alternative certification (AC) interns

who were part of the Dallas ISD’s AC program, AC interns who were part of AC programs outside the

district, certified teachers who had recently graduated from a traditional college or university program, and

certified teachers with prior experience teaching in other Texas districts. Cohort was defined as the first

school year an employee was a teacher with the Dallas ISD.1

Limitations

Sample size in subgroups was a limitation throughout the analyses. In some cases, comparisons

between TFA and non-TFA teachers were not possible due to insufficient sample sizes. For analyses of

student outcomes, only subgroups with at least two teachers were included. Though student sample sizes

were typically large enough to make statistical conclusions, a large number of students sometimes

represented a very small number of teachers. Therefore, results of subgroup analyses with small teacher

sample sizes should be interpreted with caution.

Upon consultation with representatives from TFA, 22 TFA teachers were discovered that were not

included in the Dallas ISD’s list of first-year teachers. These teachers were evenly distributed across

content areas and grade levels, therefore not representing a large portion of any particular subgroup. The

corrections from TFA were not received in time to update the current analysis but were documented to

include in future TFA analyses.

Results

Table 1 shows the number of first-year teachers in the Dallas ISD by school year cohort. The

number of first-year teachers and first-year TFA teachers increased over time.

Table 1: Number of First-Year Teachers and TFA Teachers by School Year Cohort

School YearTotal Number of1st Year Teachers

Number of 1st YearTFA Teachers

Percentage(%) of Total

2009-10 566 68 12.0

2010-11 707 97 13.72011-12 496 43 8.72012-13 1,363 84 6.22013-14 1,898 168 8.92014-15 1,124 103 9.2

Note: Number of TFA teachers included only TFA teachers found in the list of first-year teachers.

1 Cohort was determined using the teacher’s hire date into the Dallas ISD. For example, a teacher hired during thesummer preceding the 2013-14 school year or anytime during the 2013-14 school year would be included in the2013-14 cohort.


11/37


3

How does classroom effectiveness of TFA teachers compare with other

first-year teachers in the district?

Methodology

The evaluator conducted a series of Analyses of Variance (ANOVAs) to determine if differencesin teaching effectiveness existed between TFA interns and all other teachers new to the Dallas ISD each

year from 2009-10 to 2013-14. Analyses were conducted separately for Classroom Effectiveness Indices

(CEIs) in Language Arts, Mathematics, and Science for both the first and second years of teaching.2 CEIs

indicate the relative effectiveness of a teacher after adjusting for student ethnicity, language proficiency,

gender, socio-economic status, and academic performance from the previous year. Each year, CEIs are

calculated to have a mean of 50 and standard deviation of 10. Because the first four cohorts’ year 1 and

year 2 effectiveness comparisons were discrete and included the same teachers in both comparisons –

increasing the likelihood of finding a significant result when one may not exist – the evaluator selected a

statistical significance level of .025 for each statistical test (.05/2). The last cohort, 2013-14, had one year

of CEIs available and maintained a significance level of .05. The evaluator measured effect size by

calculating Eta-squared (η2), a measure of the practical influence that the type of teacher (TFA or

non-TFA) had on relative classroom effectiveness. Table 2 shows the number of TFA and Non-TFA

first-year teachers with a valid CEI score for the first and second years of teaching by subject and cohort.3

2 CEI scores are available for Foreign Language and Computer Science, but the extremely low number of teacherswith scores on those indices would not allow for valid comparisons.3 CEI scores measure teachers’ effectiveness relative to other teachers in the district each year and cannot becompared from year to year. Year 1 and year 2 CEIs were available for each cohort except the 2013-14 cohort, whichhad recently begun a second year in the Dallas ISD.


12/37


4

Table 2: Number of Teachers with Valid CEI Scores by Cohort and Subject

Cohort andSubject

Year 1 Year 2

TFA Non-TFA TFA Non-TFA

2009-10Language Arts 23 93 25 107Mathematics 18 110 21 132

Science 5 32 5 342010-11

Language Arts 33 110 30 127Mathematics 31 109 28 140Science 10 53 14 50

2011-12Language Arts 12 68 10 78Mathematics 14 67 13 76Science 11 21 10 28

2012-13Language Arts 24 179 28 218Mathematics 25 173 23 225Science 22 85 26 141

2013-14Language Arts 60 447 -- --Mathematics 59 379 -- --Science 44 236 -- --

Note: The number of teachers receiving CEI scores may vary fromyear 1 to year 2 within each cohort due to teacher attrition and thenumber of teachers meeting the criteria for CEI calculation.

Results

The results from all 27 CEI comparisons are found in Figure 1. In general, TFA teachers were

more effective than non-TFA teachers in teaching language arts, mathematics, and science in both their

first and second years. No differences among teacher-types were statistically significant in language arts.

For all three subjects, the average effectiveness of both TFA and non-TFA teachers in the first four

cohorts (2009-10 through 2012-13) improved relative to all other teachers in the district during the second

year of teaching.

Language Arts

In Language Arts, first-year TFA teachers generally received slightly higher first-year CEI scores

than their non-TFA counterparts across all cohorts except the most recent, 2013-14 cohort (see Figure 1).

None of the comparisons were statistically significant. No first-year TFA or non-TFA average scores

eclipsed 50, the average for all district teachers.Second-year TFA teachers had slightly higher CEI scores than non-TFA second-year teachers

with one exception: Non-TFA teachers in the 2009-10 cohort scored higher, on average, in their second

year than TFA teachers (M = 49.9 and M = 48.9, respectively). The TFA second-year teachers from the

2011-12 (M = 50.0) and 2012-13 (M = 50.6) cohorts scored at or above the district mean of 50. As with

the first year, differences among second-year teachers’ average Language Arts CEIs were not statistically

significant.


13/37


5

Figure 1: Average CEIs by Cohort and Content Area for TFA vs. Non-TFA Teachers

Language Arts Mathematics Science

Year Teaching (First or Second)Note: *Difference between TFA and non-TFA teachers was statistically significant (p < .025).

47.7 48.9

45.8

49.9

36

60

First Second M e a n

C E I

S c o r e

( 2 0 0 9 - 1 0 )

51.555.7

46.3

50.0

First Second*

46.7

55.7

47.6 48.2

First Second

47.9 48.947.1 48.4

36

60

First Second M

e a n

C E I S c o r e

( 2 0 1 0 - 1

1 )

53.2 52.7

46.2

49.9

First* Second

52.655.5

47.0 47.8

First Second*

46.2

50.0

45.047.1

36

60


C E I S c o r e

( 2 0 1 1 - 1

2 ) 49.7

52.2

46.948.4

First Second

50.8

57.7

39.6

46.6

First* Second*

49.4 50.6

46.548.8

36

60


C E I S c o r e

( 2 0 1 2 - 1

3 ) 50.2

55.5

46.649.4

First Second*

54.856.6

44.9

51.1

First* Second*

45.0 46.5

36

60

First M e a n

C E I S c o r e

( 2 0 1 3 - 1

4 ) 49.8

46.1

First*

TFA Non-TFA

50.8

46.4

First*


14/37


6

Mathematics

As indicated in Figure 1, TFA teachers’ mean CEI scores in Mathematics were consistently higher

than other first-year teachers across all five cohorts, ranging from 49.7 in 2011-12 to 53.2 in 2010-11.

During the 2010-11 school year, the average Mathematics CEI score for TFA interns was 7.0 points

higher than the average CEI score for other teachers. This result was statistically significant,F(1, 138) = 10.58, p = .001, and of medium effect (η2 = .07). TFA interns who began teaching during the

2013-14 school year averaged 3.7 points higher than other first-year teachers, also a statistically

significant result, F(1, 136) = 7.75, p = .006. The effect of teacher-type on effectiveness was small in this

instance (η2 = .02).

The average Mathematics CEI scores for TFA interns in their second year were higher than other

second-year teachers across all cohorts. The results were statistically significant for the 2009-10 cohort,

F(1, 151) = 5.11, p = .025. The effect size, calculated as η2, was equal to .03. This indicated that the type

of second-year teacher (TFA or non-TFA) had a small to moderate effect on effectiveness. Results for the

2012-13 cohort’s second year were also statistically significant, F(1, 246) = 8.35, p = .004, indicating asmall to moderate effect (η2 = .03). Every TFA cohort’s average CEI score was above the overall district’s

average teacher score of 50, ranging from 52.2 in 2012-13 to 55.7 in 2009-10.

Science

TFA interns were generally more effective teaching science in their first year than other types of

teachers new to the Dallas ISD, especially in the three most recent cohorts. All cohorts except the first,

2009-10 TFA cohort, averaged CEI scores above 50, meaning they were, as a group, more effective than

most teachers in the Dallas ISD. With the exception of the 2009-10 cohort, TFA interns received higher

Science CEIs on average than non-TFA teachers in their first year (see Figure 1). 4 First-year ANOVA

results were statistically significant for the 2011-12, 2012-13, and 2013-14 cohorts. During the 2011-12

cohort’s first year, TFA interns averaged 11.2 points higher than non-TFA teachers, F(1, 30) = 9.10,

p = .005. This difference indicated a large teacher-type effect (η2 = .23) on classroom effectiveness. TFA

interns who began teaching with the 2012-13 cohort scored 9.9 points higher than other first-year

teachers on average, F(1, 105) = 20.09, p < .001, with the effect size (η2) again indicative of a large

teacher-type effect by accounting for over 16 percent of the variance in effectiveness. The most recent

2013-14 cohort averaged 4.4 points higher in science than other teachers beginning that year,

F(1, 278) = 7.54, p = .006. The influence of teacher-type on effectiveness, for this cohort, could be

considered small to moderate (η2 = .03).

The second year of teaching for these cohorts yielded results similar to the first year. Across the

four second-year TFA cohorts, average Science CEI scores were above the district overall average of 50,

ranging from 55.5 to 57.7. The three most recent TFA cohorts to complete a second year – 2010-11,

2011-12, and 2012-13 – averaged effectiveness scores in science that were statistically significantly

4 It should be noted that the number of interns with valid Science CEIs in the 2009-10 cohort was small ( n = 5); thedifference was not statistically significant and should be reviewed with caution (see Table 2).


15/37


7

higher than their non-TFA second-year counterparts. The 2010-11 TFA interns averaged 7.7 points

higher than other teachers, F(1, 62) = 7.79, p = .007, a result that is associated with a medium to large

effect size (η2 = .11). The 2011-12 TFA interns scored, on average, 11.1 points higher than other

second-year teachers, F(1, 36) = 6.88, p = .013. The large effect size (η2 = .16) indicated that type of

second-year teacher accounted for over 16 percent of the variance in science classroom effectiveness.

During the 2012-13 cohort’s second year, TFA interns averaged 5.5 points higher than other teachers,

F(1, 165) = 6.07, p = .015. Teacher-type, in this cohort’s second year, had a moderate effect on

effectiveness (η2 = .04).

How does the STAAR performance of students of TFA teachers compare with

students of other teachers in the district?

Methodology

Students in grades three through eleven completed STAAR assessments in mathematics,

reading, science, and social studies, and STAAR End-of-Course (EOC) assessments in Algebra I,

English I and II, U.S. History, and Biology.5 Students were linked to teachers using the Dallas ISD’s

course files, which listed the courses each student takes and the employee ID of the teacher. The

employee ID of each student’s teacher was used to label which 2013-14 students had teachers in their

first five years of service and whether those teachers were TFA or not.6 The course file was then merged

with STAAR results to determine if there were differences in STAAR performance between students of

TFA teachers and students of non-TFA teachers.7

Two analyses were completed for each subject area and grade level. First, the percentage of

students achieving a “met expectations” rating on STAAR was compared between TFA teachers andnon-TFA teachers in their first, second, or third- through fifth-years of teaching.8 A chi-square test of

independence was used to assess whether the difference in the percentages was statistically significant.

Second, hierarchical logistic regression was used to statistically test the impact of having a TFA teacher

on achieving a “met expectations” rating on STAAR. The dependent variable was a dichotomous indicator

of “1” if a student achieved a “met expectations” rating on 2013-14 STAAR and “0” if not. The predictor

variables included prior achievement (“met expectations” rating on 2012-13 STAAR), ethnicity, sex,

5 The general STAAR testing schedule is available here: http://tea.texas.gov/student.assessment/STAAR/ 6 Only teachers in the first five years of service were included because that is how long TFA teachers have beenemployed in the district.

7 To control for the possibility that TFA teachers were placed in lower-performing schools than non-TFA teachers,only campuses with TFA teachers were included in each analysis. For example, if elementary mathematics TFAteachers were found in five campuses, the comparison group was composed of non-TFA teachers at those samecampuses.8 TFA and non-TFA teachers were compared at different levels of experience: first-year teachers, second-yearteachers, and third- through fifth-year teachers. Teachers in the third- through fifth-years of experience were groupedtogether due to small sample sizes. Subgroups were composed of different samples of teachers, not one group ofteachers tracked over time. STAAR results came from the 2013-14 school year, so results were divided among thosewho were first-year teachers in 2013-14, those who were second-year teachers in 2013-14, etc. Therefore, usecaution not to interpret differences between first-year and second-year teachers as “improvement” because the twogroups are not composed of the same teachers.


16/37


8

socioeconomic status (SES), teacher level of experience, and whether the teacher participated in TFA.

There were three models tested. In the first model, 2013-14 STAAR performance was predicted using

only student characteristics. In the second model, teacher years of experience was included to see if

including years of experience significantly improved the prediction of 2013-14 STAAR performance. In the

third model, TFA was included. If the model including TFA did a significantly better job predicting 2013-14

STAAR performance, that indicated that having a TFA teacher made a difference over and above all the

other factors considered (student characteristics, prior achievement, and teacher level of experience).

Results

STAAR Performance Overview

Figures 2 through 4 show STAAR performance for students of TFA teachers at various levels of

experience compared with students of non-TFA teachers. These percentages were further compared to

passing rates across all teachers at TFA campuses to offer perspective on how students of these

early-career TFA and non-TFA teachers performed relative to students of all teachers at the same

campuses.

Comparing across the three levels of experience and content areas, students of second-year TFA

teachers had higher passing percentages than students of first-year TFA teachers. For mathematics,

there was a trend of TFA teachers outperforming non-TFA teachers in the second year of experience.

This was also true for elementary science. However, this trend was less apparent in reading/language

arts. Small sample sizes resulted in missing data and precluded comparisons in many subgroups,

impeding trend detection. More detailed analyses of STAAR performance data by grade level and content

area follow.


17/37


9

Figure 2: Percentage That Met Expectations on Spring 2014 STAAR for Students of First-Year TFA and Non-TFA

Teachers

Figure 3: Percentage That Met Expectations on Spring 2014 STAAR for Students of Second-Year TFA and

Non-TFA Teachers

Figure 4: Percentage That Met Expectations on Spring 2014 STAAR for Students of Third- through Fifth-Year

TFA and Non-TFA Teachers

Note: Some subgroups were excluded from these figures due to small teacher sample sizes ( n < 2). Districtpercentages reflect students only in schools with TFA teachers in each subgroup. ES = Elementary School.MS = Middle School.

54 55

74

51

60

41

59

49

59

93

65

42

55

78

50

60

45 48 51

61

90

5655 56

78

5664

55 54 52

63

92

53

ES MS Alg I ES MS Eng I Eng II ES MS Bio MS

Mathematics Reading/Language Arts Science SocialStudies

P e r c e n

t T h a

t M e

t E

x p e c

t a t i o n s

o n S T A A

R

TFA Non-TFA All

64

75

86

61 6569

81

95

6255

70

55 5767 63

36

95

55 56

78

5664

55 54 52

63

92

53



P e r c e n

t T h a

t M e

t E x p e c

t a t i o n s

o n S T A A R

TFA Non-TFA All

76 7970

64

44

7162

48

78

5964

75

5355 56

78

5664

55 54 52

63

92

53



P e r c e n

t T h a

t M e

t E x p e c

t a t i o n s

o n S T A A R

TFA Non-TFA All


18/37


10

Mathematics

Figure 5 shows the percentages of elementary students that met expectations on STAAR

mathematics. Percentages were compared between first-year, second-year, and third- through fifth-year

TFA and non-TFA teachers. For all levels of experience, a statistically significant higher percentage of

elementary students of TFA teachers than non-TFA teachers passed STAAR mathematics.

Figure 5: Percentage That Met Expectations on STAAR Mathematics for Students of TFA and Non-TFA Teachers

(Grades 3 to 5)

Number of Teachers and StudentsRepresented in the Analysis

TFA Non-TFA

1st year teachers

Teachers 12 22

Students 349 814

2nd year teachers

Teachers 6 9Students 261 254

3rd-5th year teachers

Teachers 4 10

Students 115 265

Note: Asterisks indicate level of significance for chi-square test of independence *p < .05 and **p


19/37


11

Figure 6: Percentage That Met Expectations on STAAR Mathematics for Students of TFA and Non-TFA Teachers

(Grades 6 to 8)


TFA Non-TFA

1st year teachers

Teachers 10 14

Students 1,053 1,346

2nd year teachers

Teachers 6 6

Students 525 519


Teachers 1 9

Students 5 693

Note: These differences were not statistically significant. Percentage was not reported for students of third- throughfifth-year TFA teachers because there was only one teacher represented in this subgroup.

At the high school level, Figure 7 shows passing percentages on the STAAR Algebra I EOC

exam. For first-year and third- through fifth-year teacher subgroups, there was no statistically significant

difference between TFA teachers and non-TFA teachers in percentage of students passing. For

second-year teachers, a higher percentage of students of TFA teachers (86%) passed STAAR

mathematics compared with students of non-TFA teachers (55%). The difference was statistically

significant.

Figure 7: Percentage That Met Expectations on STAAR Algebra 1 EOC for Students of TFA and Non-TFA

Teachers


TFA Non-TFA

1st year teachers

Teachers 14 4

Students 1,038 248

2nd year teachers

Teachers 6 3

Students 472 152


Teachers 2 3

Students 191 235

Note: Asterisk indicates level of significance for chi-square test of independence *p < .001.

Logistic regression results for elementary STAAR mathematics are shown in Table 3. The results

indicated that prior achievement, teacher experience level, and TFA status were associated with

55%

75%

55%

70%

48%

1st year teachers 2nd year teachers 3rd-5th year teachers

TFA Non-TFA

n/a

74%86% 79%78%

55%

78%

1st year teachers 2nd year teachers* 3rd-5th year teachers

TFA Non-TFA


20/37


12

significantly greater likelihood of passing STAAR mathematics.9 The odds ratio for TFA status indicated

that elementary students of TFA teachers were 3.228 times more likely to pass STAAR mathematics than

students of non-TFA teachers when accounting for teacher experience and student characteristics. The

addition of TFA status in Model 3 resulted in a statistically significant improvement in overall model fit,10

indicating that TFA status had an impact on STAAR performance over and above other student and

teacher characteristics in the model.

Across all teachers (TFA and non-TFA), compared with Hispanic students, students who were

African American (odds ratio = 0.540) or Other Ethnicity (odds ratio = 0.159) were significantly less likely

to pass STAAR. According to the odds ratio, students who passed STAAR mathematics in 2012-13 were

13.112 times more likely to pass STAAR in 2013-14. Compared with first-year teachers, students of

second-year teachers were 2.179 times more likely to pass STAAR, and students of third- through

fifth-year teachers were 2.645 times more likely to pass STAAR.

Table 3: Logistic Regression Final Model Results for STAAR Mathematics Grades 4 to 5

Predictor B z Odds RatioSTAAR Mathematics Met Expectations 2012-13 2.574 282.910** 13.112

African American -0.615 12.898** 0.540Other Ethnicity -1.838 4.048* 0.159Male 0.084 0.344 -Low SES -0.712 2.671 -Second-year teacher 0.779 19.067** 2.179Third- through fifth-year teacher 0.973 25.003** 2.645TFA 1.172 48.693** 3.228

Note: Final model Nagelkerke R2=.431. *p < .05 and **p


21/37


13

2013-14 when teacher experience and student characteristics were held constant. Compared with first-

year teachers, students of second-year teachers were 2.169 times more likely pass STAAR. However,

students of third- through fifth-year teachers were less likely than students of first-year teachers to pass

STAAR (odds ratio = 0.765) when teacher experience and student characteristics were held constant.

Table 4: Logistic Regression Final Model Results for STAAR Mathematics Grades 6 to 8

Predictor B z Odds Ratio

STAAR Mathematics Met Expectations 2012-13 2.105 739.687** 8.205 African American -0.743 65.277** 0.476Other Ethnicity -0.189 0.356 -Male -0.054 0.501 -Low SES -0.006 0.001 -Second-year teacher 0.774 65.241** 2.169Third- through fifth-year teacher -0.269 6.036* 0.765TFA 0.207 5.906* 1.230



22/37


14

Mathematics Results Summary

At the elementary school level, a greater percentage of students of TFA teachers of all

experience levels passed the STAAR mathematics test. For Algebra I students, a greater percentage of

students of second-year TFA teachers passed the Algebra I EOC exam. There were no other significant

differences in passing rates between TFA and non-TFA teachers. According to the logistic regression

results, having a TFA teacher significantly improved the odds of a student passing STAAR mathematics,

ranging from odds 1.2 times higher for middle school students to more than three times higher for

elementary students after holding teacher experience and student characteristics constant. Students of

non-TFA teachers did not statistically significantly outperform students of TFA teachers in any of the

comparisons reported above.

Reading/Language Arts

Figure 8 shows passing percentages on STAAR reading for students in grades three through five.

For first-year and second-year teacher subgroups, there was no statistically significant difference

between TFA teachers and non-TFA teachers in percentage of students passing. For third- through

fifth-year teachers, a statistically significant higher percentage of students of TFA teachers (70%) passed

STAAR reading compared with students of non-TFA teachers (59%).

Figure 8: Percentage That Met Expectations on STAAR Reading for Students of TFA and Non-TFA Teachers

(Grades 3 to 5)


TFA Non-TFA

1st year teachers

Teachers 19 33

Students 662 1,058

2nd year teachers

Teachers 4 15

Students 173 481


Teachers 4 14

Students 91 437

Note: Asterisks indicate level of significance for chi-square test of independence *p < .05.

Figure 9 shows passing percentages on STAAR reading for students in grades six through eight.

For all teacher subgroups, there was no statistically significant difference between TFA teachers and

non-TFA teachers in percentage of students passing.

51%

61%

70%

50%

57% 59%

1st year teachers 2nd year teachers 3rd-5th year teachers*

TFA Non-TFA


23/37


15

Figure 9: Percentage That Met Expectations on STAAR Reading for Students of TFA and Non-TFA Teachers

(Grades 6 to 8)


TFA Non-TFA

1st year teachers

Teachers 9 33

Students 621 2,807

2nd year teachers

Teachers 13 14

Students 1,020 767


Teachers 4 12

Students 301 689

Note: These differences were not statistically significant.

Figure 10 shows passing percentages on the STAAR English I EOC assessment. There was no

statistically significant difference between first-year TFA teachers and non-TFA teachers in percentage of

students passing. However, the percentage of students passing was statistically significantly higher for

students of third- through fifth-year non-TFA teachers (75%) than students of TFA teachers (44%).

Figure 10: Percentage That Met Expectations on STAAR English 1 EOC for Students of TFA and Non-TFA

Teachers


TFA Non-TFA

1st year teachers

Teachers 4 5

Students 363 274

2nd year teachers

Teachers 0 1

Students 0 133


Teachers 2 2

Students 200 201

Note: Asterisks indicate level of significance for chi-square test of independence *p < .001. Percentage is notreported for students of second-year teachers because there was only one non-TFA teacher and no TFA teachers

represented in this subgroup.

Figure 11 shows passing percentages on the STAAR English II EOC assessment. A significantly

greater percentage of students of first-year TFA teachers passed the English II EOC (59%) than students

of non-TFA teachers (48%). Sixty-three percent of students of non-TFA second-year teachers passed the

English II EOC. Comparisons between TFA and non-TFA were not possible for second- or third- through

fifth-year teachers due to small teacher sample sizes.

60%65% 64%

60%67% 64%


TFA Non-TFA

41% 44%45%

75%

1st year teachers 2nd year teachers 3rd-5th year teachers*

TFA Non-TFA

n/a


24/37


16

Figure 11: Percentage That Met Expectations on STAAR English 2 EOC for Students of TFA and Non-TFA

Teachers


TFA Non-TFA

1st year teachersTeachers 4 5

Students 354 302

2nd year teachers

Teachers 1 2

Students 93 209


Teachers 1 1

Students 52 118

Note: Asterisks indicate level of significance for chi-square test of independence *p < .05. Some percentages werenot reported due to small sample sizes.

Logistic regression results for elementary STAAR reading are shown in Table 6. The results

indicated that prior achievement, teacher experience level, and TFA participation were associated with

significantly greater likelihood of passing STAAR reading.13 Elementary students of TFA teachers had

1.607 times higher odds of passing STAAR reading than students of non-TFA teachers after holding

teacher experience and student characteristics constant. The addition of TFA status in Model 3 resulted

in a statistically significant improvement in overall model fit,14 indicating that TFA status had an impact on

STAAR performance over and above other student and teacher characteristics in the model.

Across all teachers (TFA and non-TFA), students who passed STAAR reading in 2012-13 were

11.869 times more likely to pass STAAR reading in 2013-14 when teacher experience and student

characteristics were held constant. The odds of students of second-year teachers passing the STAAR

were 1.585 times higher than the students of first-year teachers, but students of third- through fifth-year

teachers were not significantly more or less likely to pass STAAR when holding teacher experience and

student characteristics constant. Compared with Hispanic students, students who were African American

were significantly less likely to pass STAAR (odds ratio = 0.623).

13 Odds ratios greater than one indicated the predictor is associated with greater likelihood of passing STAAR. Oddsratios less than one indicated the predictor is associated with less likelihood of passing STAAR.14 Addition of TFA status in Model 3 increased Nagelkerke R2 from .376 (Model 2) to .383. Model fit improvement asmeasured by -2 log likelihood was statistically significant (p < .001).

59%48%

63%

1st year teachers* 2nd year teachers 3rd-5th year teachers

TFA Non-TFA

n/a n/a


25/37


17

Table 6: Logistic Regression Final Model Results for STAAR Reading Grades 4 to 5


STAAR Reading Met Expectations 2012-13 2.474 413.799** 11.869 African American -0.474 13.533** 0.623Other Ethnicity 0.765 2.087 -Male -0.222 3.498 -

Low SES -0.270 0.621 -Second-year teacher 0.460 8.259* 1.585Third- through fifth-year teacher -0.022 0.019 -TFA 0.474 12.222** 1.607



26/37


18

experience and student characteristics constant. The addition of TFA status in Model 3 resulted in a

statistically significant improvement in overall model fit,15 indicating that TFA status had an impact on


Across all teachers (TFA and non-TFA), students who passed STAAR reading in 2012-13 had

9.537 times higher odds of passing the STAAR English I EOC in 2013-14. Compared with first-year

teachers, students of third- through fifth-year teachers were 1.463 times more likely pass STAAR when

student characteristics were held constant. No other student or teacher characteristics were significantly

associated with passing the English I EOC.

Table 8: Logistic Regression Final Model Results for STAAR English I End-of-Course Exam


STAAR Reading Met Expectations 2012-13 2.255 138.101** 9.537 African American 0.085 0.220 -Other Ethnicity -0.250 0.371 -Male -0.229 2.671 -Low SES -0.101 0.211 -Second-year teacher -0.093 0.150 -Third- through fifth-year teacher 0.380 5.694* 1.463TFA -0.711 21.421** 0.491



27/37


19

Table 9: Logistic Regression Final Model Results for STAAR English II End-of-Course Exam


STAAR EOC Reading I Met Expectations 2012-13 2.308 234.116** 10.056 African American -.876 22.356** 0.417Other Ethnicity .116 0.039 -Male -.575 14.942** 0.563

Low SES .285 1.588 -Second-year teacher .241 1.931 -Third- through fifth-year teacher .577 6.892* 1.781



28/37


20

Figure 12: Percentage That Met Expectations on STAAR Science for Students of TFA and Non-TFA Teachers

(Grade 5)


TFA Non-TFA

1st year teachers

Teachers 5 6

Students 179 231

2nd year teachers

Teachers 2 4

Students 109 139


Teachers 0 5

Students 0 146

Note: Asterisks indicate level of significance for chi-square test of independence *p < .05. Some percentages werenot reported due to small sample sizes.

Figure 13 shows passing percentages on STAAR science for students in grade eight. There was

no statistically significant difference between first-year TFA teachers and non-TFA teachers in percentage

of students passing. For second-year TFA teachers, 81 percent of students passed STAAR science,

while 71 percent of third- through fifth-year TFA teachers passed. There were no non-TFA second or

third- through fifth-year teachers.

Figure 13: Percentage That Met Expectations on STAAR Science for Students of TFA and Non-TFA Teachers

(Grade 8)


TFA Non-TFA

1st year teachers

Teachers 4 2

Students 422 209

2nd year teachers

Teachers 2 0

Students 260 0


Teachers 2 0

Students 257 0

Note: These differences are not statistically significant. Some percentages were not reported due to small samplesizes.

Figure 14 shows passing percentages on the STAAR Biology EOC. There was no statistically

significant difference for first-year or second-year TFA teachers and non-TFA teachers in percentage of

students passing. There were no non-TFA third- through fifth-year teachers to compare with TFA

teachers.

49%

69%

51%

36%

53%

1st year teachers 2nd year teachers* 3rd-5th year teachers

TFA Non-TFA

59%

81%

71%

61%


TFA Non-TFA

n/an/a

n/a


29/37


21

Figure 14: Percentage That Met Expectations on STAAR Biology EOC for Students of TFA and Non-TFA Teachers


TFA Non-TFA

1st year teachers

Teachers 6 7

Students 660 586

2nd year teachers

Teachers 7 4

Students 595 408


Teachers 1 0

Students 79 0

Note: These differences are not statistically significant. Some percentages were not reported due to small samplesizes.

Logistic regression results for STAAR science for grade five are shown in Table 10. The results

indicated that prior achievement, teacher experience level, and TFA status were associated with

significantly greater likelihood of passing STAAR science, while no predictors were associated with less

likelihood of passing. The odds ratio for TFA status indicated that fifth-grade students of TFA teachers

had 1.684 times higher odds of passing STAAR science than students of non-TFA teachers when holding

teacher experience and student characteristics constant. The addition of TFA status in Model 3 resulted

in a statistically significant improvement in overall model fit,16 indicating that TFA status had an impact on


Across all teachers (TFA and non-TFA), students who passed STAAR mathematics in 2012-1317

were 6.158 times more likely to pass STAAR science in 2013-14. Compared with first-year teachers,

students of third- through fifth-year teachers were 1.632 times more likely pass STAAR when student

characteristics were held constant. No other student or teacher characteristics were significantly

associated with passing STAAR science at grade five.

16 Addition of TFA status in Model 3 increased Nagelkerke R2 from .226 (Model 2) to .238. Model fit improvement asmeasured by -2 log likelihood was statistically significant (p < .01).17 STAAR mathematics was used as an indicator of prior achievement for STAAR science in grades five and eightbecause no STAAR science assessment was administered to students in grades four and seven to serve as priorachievement. For Biology EOC, which is administered primarily to students in grade nine, grade eight STAAR sciencewas used as prior achievement.

93% 95%90% 95%


TFA Non-TFA

n/a


30/37


22

Table 10: Logistic Regression Final Model Results for STAAR Science Grade 5


STAAR Mathematics Met Expectations 2012-13 1.818 120.019** 6.158 African American 0.153 0.591 -Other Ethnicity 2.245 3.776 -Male 0.037 0.052 -

Low SES -0.613 1.642 -Second-year teacher 0.059 0.101 -Third- through fifth-year teacher 0.490 4.152* 1.632TFA 0.521 8.215* 1.684



31/37


23

that TFA status did not have an impact on STAAR performance over and above other student and

teacher characteristics in the model.

Across all teachers (TFA and non-TFA), the odds of students who passed STAAR science in

2012-13 also passing the STAAR Biology EOC in 2013-14 were 6.099 times higher than those who did

not pass STAAR science in 2012-13 when student characteristics and teacher experience were held

constant. Compared with females, males were less likely to pass the Biology EOC exam

(odds ratio = 0.651). Holding student characteristics constant, students of second-year teachers were

1.732 times more likely to pass STAAR when compared with students of first-year teachers, No other

student or teacher characteristics were significantly associated with passing STAAR science.

Table 12: Logistic Regression Final Model Results for STAAR Biology End-of-Course Exam


STAAR Science Met Expectations 2012-13 1.808 72.425** 6.099 African American -0.034 0.018 -Other Ethnicity -0.542 1.801 -

Male -0.429 5.233* 0.651Low SES -0.065 0.056 -Second-year teacher 0.549 7.673* 1.732Third- through fifth-year teacher 18.178 .000 -



32/37


24

Figure 15: Percentage That Met Expectations on STAAR Social Studies for Students of TFA and Non-TFA

Teachers (Grade 8)


TFA Non-TFA

1st year teachers

Teachers 4 6

Students 382 577

2nd year teachers

Teachers 2 1

Students 252 115


Teachers 0 1

Students 0 129

Note: Asterisks indicate level of significance for chi-square test of independence *p < .01.

Logistic regression results for STAAR social studies for grade eight are shown in Table 13. The

results indicated that prior achievement, being an ethnicity other than Hispanic or African American, being

male, and TFA status were associated with significantly greater likelihood of passing STAAR social

studies, while teacher experience level was associated with less likelihood of passing. The odds ratio for

TFA status indicated that eighth grade students of TFA teachers had 1.622 times higher odds of passing

STAAR social studies than students of non-TFA teachers when controlling for teacher experience and

certain student characteristics. The addition of TFA status in Model 3 resulted in a statistically significant

improvement in overall model fit,18 indicating that TFA status had an impact on STAAR performance over

and above other student and teacher characteristics in the model.

Across all teachers (TFA and non-TFA), students who passed STAAR reading in 2012-1319 were

9.307 times more likely to pass STAAR social studies in 2013-14. Students of ethnicity other than

Hispanic or African American were 2.363 times more likely to pass STAAR social studies. Males were

1.391 times as likely as females to pass STAAR social studies. Compared with first-year teachers,

students of second-year teachers were less likely to pass STAAR social studies (odds ratio = 0.726)

when student characteristics were held constant. No other student or teacher characteristics were

significantly associated with passing STAAR social studies at grade eight.

18 Addition of TFA status in Model 3 increased Nagelkerke R2 from .283 (Model 2) to .292. Model fit improvement asmeasured by -2 log likelihood was statistically significant (p < .001).19 STAAR reading was used as an indicator of prior achievement for STAAR social studies in grade eight because noSTAAR social studies assessment is administered to students in grade seven to serve as prior achievement.

65% 62%56%

1st year teachers* 2nd year teachers 3rd-5th year teachers

TFA Non-TFA

n/a n/a


33/37


25

Table 13: Logistic Regression Final Model Results for STAAR Social Studies Grade 8


STAAR Reading Met Expectations 2012-13 2.231 227.777** 9.307 African American -0.103 0.327 -Other Ethnicity 0.860 8.369* 2.363Male 0.330 6.691* 1.391

Low SES -0.244 1.737 -Second-year teacher -0.320 4.456* 0.726Third- through fifth-year teacher -0.425 3.562 -TFA 0.483 12.273** 1.622



34/37


26

90.7 percent for the 2011-12 cohort to 100.0 percent of the first, 2009-10 cohort. Of the four TFA cohorts

with at least three school years in the district, third-year retention rates ranged from 33.0 percent for the

2010-11 cohort to 51.5 percent for the 2009-10 cohort. As for non-TFA cohorts, third-year retention rates

have dropped each year since 2009-10. Over 74 percent of the 2009-10 first-year non-TFA teachers

began a third year. The 2012-13 cohort returned 57.6 percent of its non-TFA members. Specific cohort

results follow.

The first cohort of TFA interns began teaching during the 2009-10 school year. Figure 16 shows

the percentage of TFA and non-TFA first-year teachers in the 2009-10 cohort who began the subsequent

school year as a teacher with the Dallas ISD. Every TFA intern (n = 68) returned for a second year

compared with 86.7 percent of other first-year teachers (n = 432). Precisely 51.5 percent of the TFA

teachers began a third year (n = 35) while 74.5 percent of other teachers did so (n = 371). The trend

continued through the beginning of the sixth year, with 14.7 percent of the original TFA teachers still with

the Dallas ISD (n = 10). Exactly 39.4 percent (n = 196) of other teachers continued teaching with the

Dallas ISD.

Figure 16: 2009-10 Cohort Retention Rate by Year Began

Figure 17 illustrates the retention rates for the 2010-11 TFA and non-TFA teachers beginning a

second through fifth year with the Dallas ISD. TFA teachers returned for a second year at slightly higher

rates than other second-year teachers (n = 92, 94.8% and n = 525, 86.0%, respectively). Of the original

610 non-TFA teachers starting with the 2010-11 cohort, 414 (67.9%) began a third year as a teacher with

the Dallas ISD. Comparatively, 33.0 percent (n = 32) of the TFA teachers began a third year.

0

10

20

30

40

50

60

70

80

90

100

2nd 3rd 4th 5th 6th

P e r c e n

t R e

t a i n e

d

Year Began

TFA

Non-TFA


35/37


27


As is shown in Figure 18, the trend of TFA teachers returning for a second year at higher rates

(n = 39, 90.7%) than other teachers was also true for the 2011-12 cohort (n = 367, 81.0%). However, as

with the previous cohorts, a lower proportion of TFA teachers (n = 17, 39.5%) than other types of

teachers (n = 269, 59.4%) began a third year in the classroom.


After their first year as teachers in the Dallas ISD during the 2011-12 school year, 95.2 percent of the TFA

teachers returned a second year (n = 80); 37 of the original 84 interns – 44.0 percent – began a third year

(see Figure 19). Whereas 1,279 non-TFA teachers started with the Dallas ISD during the 2011-12 school

year, exactly 946 (73.4%) returned a second year, a proportion lower than previous cohorts; 57.6 percent

of non-TFA teachers returned a third year (n = 737).

0

10

20

30

40

5060

70

80

90

100

2nd 3rd 4th 5th

P e r c e n

t R e

t a i n e

d

Year Began

TFA

Non-TFA

0

10

20

3040

50

60

70

80

90

100

2nd 3rd 4th

P e r c e

n t R e

t a i n e

d

Year Began

TFA

Non-TFA


36/37


28


As shown in Figure 20, of the 168 TFA interns who began teaching during the 2013-14 school

year, 157 returned to teach a second year (93.5%). Comparatively, 73.6 percent of the 1,730 non-TFA

teachers returned to Dallas ISD classrooms (n = 1,274).

Figure 20: 2013-14 Cohort Second-Year Retention

SUMMARY AND RECOMMENDATIONS

Summary

Since the first cohort of TFA interns began with the 2009-10 school year, 563 interns have taught

in Dallas ISD schools. The number of teachers in each cohort has increased over time.

Classroom Effectiveness Indices

The results of a series of ANOVAs determined that TFA teachers were generally more effective,

on average, than non-TFA teachers in teaching language arts, mathematics, and science for both their

first and second years. The differences in average mathematics and science effectiveness were

statistically significant for most cohorts in either their first year, second year, or both years teaching. All

0

10

20

30

40

5060

70

80

90

100

2nd 3rd

P e r c e n

t R e

t a i n e

d

Year Began

TFA

Non-TFA

0

10

20

30

40

50

60

70

80

90

100

2nd

P e r c e n

t R e

t a i n e

d

Year Began

TFA

Non-TFA


37/37


TFA cohorts averaged effectiveness ratings above 50 in their second year teaching, which was higher

than the average for all teachers in the Dallas ISD.

Student

STAAR

Performance

To determine the impact of having a TFA teacher on students meeting the STAAR passing

standards, a series of hierarchical logistic regressions was conducted for mathematics, reading/language

arts, science, and social studies. Analyses were conducted separately for elementary, middle, and high

school students.

For mathematics, students of TFA teachers statistically significantly outperformed students of

non-TFA teachers across all levels when teacher experience and student characteristics were held

constant. Odds ratios indicated that elementary students of TFA teachers were 3.23 times more likely

than students of non-TFA teachers to meet expectations. Middle school students of TFA teachers were

1.23 times more likely, and Algebra I students of TFA teachers were 1.63 times more likely than students

of non-TFA teachers to meet STAAR passing standards.

The results for reading/language arts were mixed. Having a TFA teacher positively impacted the

odds of elementary students passing the STAAR, had no impact for middle school and English II

students, and negatively impacted the odds of English I students meeting the minimum standard. Science

results were also mixed. After accounting for student characteristics and teacher experience, having a

TFA teacher increased the odds of passing the fifth-grade science STAAR by 1.6. TFA was not a

predictor of passing rates for grade 8 science or the Biology EOC exams. Grade 8 social studies students

with a TFA teacher were 1.62 times more likely to meet STAAR expectations than students with a

non-TFA teacher.

Teacher Retention

TFA interns in each cohort have returned to teach a second year with the Dallas ISD at rates

ranging from 90.7 to 100.0 percent. For each cohort, these rates were higher than the rates of non-TFA

teachers. Because TFA requires a two-year teaching commitment, smaller proportions of TFA teachers

returned for a third year when compared to non-TFA teachers. Overall, the proportion of all beginning

TFA and non-TFA teachers returning for a third year has declined with each cohort since 2009-10.

Recommendation

Find ways to encourage TFA teachers to stay in the Dallas ISD classrooms beyond their

two-year commitment. TFA teachers in the Dallas ISD have historically been effective classroom

teachers. Results from the CEI analyses indicated that second-year TFA teachers have been, on

average, more effective teaching mathematics and science than other teachers in the district. The logistic

regression analyses revealed that mathematics students of TFA teachers at all grade levels had

increased odds of meeting the 2013-14 STAAR standards when accounting for student characteristics

and teacher experience. The district should encourage TFA teachers to return to the classroom for a third

year and beyond.

teach for america study 2015

Documents