teach for america study

teachforamerica

Case study of effeCtivenessAnalysis and Compilation by

the sChultz Center

for teaChing and leadership

Jacksonville, Florida · July 2011

teachforamerica

teach for impactfor leadership

teach for change teach for all of Us teach for the fUtUre teach for transformation

teach for family for opportUnity teach for impactfor leadership

teach for change teach for all of Us teach for the

Teach for america (Tfa) is a nonprofit organization committed to placing top recent college graduates into local teaching forces of beginning teachers currently serving in urban and rural school districts across the country. as part of their commitment to work as Tfa corps members, these teachers agree to teach for two years in public schools, and in the process it is hoped that these individuals evolve into strong instructional leaders who seek to expand educational opportunities for all students. Teach for America – Jacksonville currently has corps members placed in 23 Duval county schools at the elementary, middle, and high school levels. in march 2011, Teach for America – Jacksonville requested that an external analysis be done of current performance by corps members serving in Duval county schools. The Schultz center for Teaching and Leadership was asked to provide this third party review of available data to inform Tfa work in several ways: • To provide Tfa teachers, staff members, and

stakeholder groups with relevant information about the progress made to date by corps members in comparison with other novice teachers in Duval county schools

• To make information available about various sources of student data and their relative utility in offering teachers and Tfa staff members with reliable data for decision making about performance of students; and

• To make recommendations for an ongoing plan for data collection and analysis that will allow Tfa and the Duval county School Board with timely and relevant information about the progress of students and the effectiveness of Teach for america corps members.

Because Teach for america corps members are at work in elementary, middle, and high schools, student achievement data from all levels was reviewed with the goal of answering the following question: How does the student performance of students in TFA classrooms compare with that of similar experience level teachers in non-TFA classrooms?

Who are we?

Tennessee state report finds that Teach For America corps members outperform their peersteachforamerica

While student achievement in all areas is important, the most commonly cited indicators of student proficiency discussed widely in florida are those found on the florida comprehensive assessment Tests (fcaT) in grades 3-10 in reading and mathematics. These state-mandated accountability measures were used for most of the analyses conducted for this report. in addition, we included a recently developed assessment of reading at the primary grade levels called fair (florida assessments for instruction in reading) to allow us to include data for Tfa corps members assigned to primary grade levels. in all, we were able to include 101 corps members, among whom about one half were serving in grade levels and subject area assignments which permitted the inclusion of student performance data allowing for the analysis of student gains. The question at hand was whether the student gains of Tfa corps members compared favorably with that of all novice (beginning) teachers, and in particular, the group of beginning teachers placed in similar schools. after clarifying questions of Tfa staff members and discussing reasonable comparison groups that might be used to answer questions about efficacy, we arrived at several primary groups which would serve as the source of contrasts made in the analysis of student achievement gains: • Students in Tfa classrooms • Students in Non-Tfa classrooms (in the same

schools) • Students in Novice (Non-Tfa) classrooms • Students in Title i Schools • Students in Turnaround Schools • Students in Duval county (Sample)Student achievement gains available for students in Grade 2 (fair), grades 4-8 (fcaT) in reading, and grades 3-10 (fcaT) in mathematics were used to carry out statistical analyses on differences that were found among the gains made by grade level and by named comparison groups of teachers. Because state percentile norms do not exist for the fcaT, all Duval county scores were ranked to create “local norms” and percentiles which could then be used

What did we do?

Teach For America is one of only ten new organizations on FORTUNE’s list of 100 Best Companies to Work For

to compare the relative gains made by teachers in various groups. Local percentile ranks were assigned to all students in Grade 2 for whom fair Probability of Reading Success (PrS) scores were available (n = 8700) and for all students at grades 4 - 10 in reading and mathematics for whom previous year and current year data created viable “gain scores” (approximately 60,000 scores in reading and 76,000 in mathematics). Local percentile ranks were calculated by standardizing gain scores and ranking them within grade levels. allowing for the uniqueness of grade level patterns of test content and difficulty, the rank ordering of gains within grade level would give us a good representation of what “typical” growth looks like currently in our schools. Based on the percentile ranking process in reading and mathematics, gains produced for all students were then sorted into four quartile groupings: • Quartile 1: 1st through 25th local percentiles • Quartile 2: 26th through 50th local percentiles • Quartile 3: 51st through 75th local percentiles • Quartile 4: 76th through 99th local percentilesUsing the placement of student gains within local percentile ranks and quartiles, we can determine whether students made growth placing them in the 1st through the 4th quartile of gains seen in Duval county. This process allowed us to use commonly available measures (fair and fcaT) in reading and mathematics to qualify the relative size of gains made by students of teachers within various comparison groups. The average (or typical) performance in Duval county was determined by local percentiles calculated with all students, so variations would occur randomly across all. if gains differ significantly by subgroups of teachers identified by other factors, then, this may provide evidence that the effectiveness of these groups may similarly differ. Based on these sources of information, we were able to make distinctions in two ways: 1. average gains of students made in classrooms of Tfa

teachers with other comparison group teachers. 2. Proportions of student gains falling in the top (3rd

and 4th) local quartiles of performance (i.e., above average performance).

Harvard study finds that Teach For America is producing a significant number of education entrepreneurs

What did we learn?

following are results tables reflecting comparisons of fair PrS score and fcaT gains made by students in classrooms of comparison groups in contrast with student results in Teach for america classrooms. Top Quartile (3rd and 4th Quartile) Gains were based on local percentile ranks calculated on Duval county total group scores in reading and mathematics. cells in these tables are shaded to reflect findings using tests of statistical significance:

Green = statistically significant results in a positive direction (+)

Yellow = no significant difference or comparable results (≈)

Blue = Duval county Sample (random sample of all students)

Gray = indicates the teacher comparison group used in statistical analysis

NBC’s TODAY show recently featured Wendy Kopp’s Washington Post bestseller, A Chance to Make History, and TFA alumnus Brett Kimmel

“ I am indebted to my father for living, but to my teacher for living well.”

a L e x a N D e r T h e G r e aT

What did we learn?

READiNg gAiNs iNlOCAl 3RD & 4TH QUARTilEs

Grade 2 Count & % w/in grade






Novice T1

Teachers

17455.8%

15732.5%

15940.0%

15830.3%

11633.1%

14131.2%

Novice Teachers

27852.6%

30041.1%

30147.0%

26338.1%

24241.1%

22038.6%

TASchools

20155.2%

25040.6%

21241.3%

9522.0%

7526.1%

10628.3%

Title 1 Schools

29354.7%

40642.4%

34744.1%

32240.6%

29245.4%

32443.1%

Duval County sample

192550.0%

107750.0%

101250.0%

70050.0%

61250.0%

65150.0%

Number/percent of students top quartile gains

TFATeachers

5775.3%

10354.2%

4040.8%

12444.8%

8152.3%

9944.4%

Reading Comparisons by Teacher and school groups.

Reading summary statements: Percent of students in Tfa classrooms making gains which were above average locally (Q3 & Q4):

• Were significantly higher at Grade 2 (75.3%) than other comparison groups and the district as a whole (+).

• Were also significantly higher than comparison groups at grade 4, 6, 7, and 8 (+).

• Were not significantly different at Grade 5.• improvement most needed at grades 5, 6, and 8.

5,200 new Teach For America teachers recently joined the national effort to expand educational opportunity

mATH gAiNs iNlOCAl 3RD & 4TH QUARTilEs








Novice T1

Teachers

17456.3%

10944.7%

14757.6%

16559.9%

16455.8%

12660.9%

13554.8%

Novice Teachers

27852.2%

18148.2%

21957.3%

24956.1%

20254.2%

21253.0%

19049.5%

TASchools

20158.0%

15945.3%

17861.2%

14059.6%

17054.8%

27956.0%

28552.1%

Title 1 Schools

29356.2%

24647.9%

33255.2%

29856.2%

32654.7%

25856.3%

26653.0%

Duval County sample

58950.0%

55250.0%

53750.0%

51450.0%

55350.0%

44550.0%

44650.0%

Number/percent of students top quartile gains

TFATeachers

5752.3%

5745.2%

9166.9%

4270.0%

9264.4%

3969.6%

7456.4%

math Comparisons by Teacher and school groups.

math summary statements: Percent of students in Tfa classrooms making gains which were above average locally (Q3 & Q4):

• Were significantly higher at grades 6, 7, 8, and 9 than other comparison groups (+).

• Were not significantly different at grades 4, 5, and 10. • improvement most needed at Grade 5.

in conclusion.

The goal of this study was to determine the impact of Tfa teachers on student growth in reading and mathematics as compared with that of similar experience teachers in non-Tfa classrooms. The answer must also be put in context as it relates to “customer satisfaction” and district “staying power”. reviewing student gains, it’s clear that Tfa corps members are having a positive impact on student achievement in Duval county. among 13 individual statistical comparisons made between students in Tfa classes with those of other Novice Title i classrooms, 9 were found to be statistically significant in a positive direction, while 4 were found to be comparable. in only one instance (4th grade math) did the proportion of students making above average gains (locally) fall below that of the most similar comparison group. These findings mirror national studies which demonstrated that student achievement under corps members was great or greater than that of other new teacher groups. Principals in schools where Tfa corps members were assigned provided an important source of information about the quality of corps members. The goal of ensuring that effective teachers are placed in every classroom is critical. Duval county principals expressed high levels of satisfaction on a survey completed in 2009: • 92% said that Tfa teachers ‘have made a positive

difference in the school environment’ • 75% rated the preparation given to Tfa teachers as

‘above average’ • 92% would recommend hiring of Tfa teachers There is a concern about the commitment and staying power of corps members in Duval county. By the close of school year 2009-10, 109 corps members had been placed in schools. among those, 89 completed two-year commitments. Within that group, 26 continued teaching in Duval county, and 63 did not. of that 63, 45 stayed in the field of education. in total, 71 of the 89 (80%) who completed their two-year commitment are still involved in education. efforts are ongoing to enhance the Tfa corps members commitment to Jacksonville.

Teach For America’s 20th Anniversary summit draws nearly 11,000 committed to closing the achievement gap

teachforamerica

teach for impactfor leadership

teach for change teach for all of Us teach for the fUtUre teach for transformation

teach for family for opportUnity teach for impactfor leadership

teach for change teach for all of Us teach for the

4019 Boulevard center Drive

Jacksonville, florida 32207

904 348 7239

www.schultzcenter.org

aBoUT The aUThor: Katherine Divine, Ph.D. is an experienced assessment and evaluation specialist currently serving in a shared position with Duval county Public Schools as Executive Director for Research Design and Evaluation at the Schultz center. her past experience includes tenure as a Research Associate at the University of North florida (florida institute of education); Director of Research, Evaluation and Accountability for the District School Board of Pasco county; and Director of Research, Communications, and Student Services for Portsmouth Public Schools in Virginia. for more information about how programs are evaluated for effectiveness, or for other reports and publications, visit www.schultzcenter.org.

teach for america study

Documents