unc teacher quality research: 2013 teacher preparation ... · unc teacher quality research: 2013...
Post on 09-Oct-2020
6 Views
Preview:
TRANSCRIPT
UNC Teacher Quality Research:2013 Teacher Preparation
Program Effectiveness Report- MAY 2013 -
Gary T. Henry, Vanderbilt University
Kristina M. Patterson, UNC–Chapel Hill
Shanyce L. Campbell, UNC–Chapel Hill
Pan Yi, UNC–Chapel Hill
UNC Teacher Quality Research:
2013 Teacher Preparation Program Effectiveness Report
May 2013
Gary T. Henry, Vanderbilt University
Kristina M. Patterson, UNC–Chapel Hill
Shanyce L. Campbell, UNC–Chapel Hill
Pan Yi, UNC–Chapel Hill
Contents
Acknowledgements ………………………….………………………… i
Introduction ………………………………………………….………. 1
Data and Methods……..…….………………………………….…….. 3
Results ……………………………………………………………... 8
Conclusion ………………………………………………………….. 17
Calculating Days Equivalency ……………………………………….….. 18
Tables:
Table 1: Standard Model Control Variables ……………………………….. 4
Table 2: Institution Abbreviations ……………………………………….. 5
Table 3: UNC Institutional Counts ………………………………….…… 7
Table 4: Summary of Results ………………………………….….….… 15
Table 5: Teacher Counts by UNC System School, Level and Subject …………… 16
Table 6: Key for the Interpretation of Coefficients (Days Equivalency) ……..…… 17
Figures:
Figure 1: UNC Programs vs. All Other Sources of Teachers: Elementary School ……. 12
Figure 2: UNC Programs vs. All Other Sources of Teachers: Middle School ……..… 13
Figure 3: UNC Programs vs. All Other Sources of Teachers: High School …….…… 14
i
Acknowledgements
We wish to recognize Alisa Chapman and Keith Brown with the University of North
Carolina General Administration for their vital contributions in providing data and working as
partners throughout the research and communication processes.
We also wish to thank the deans and department heads from the colleges, schools and
departments of education at the 15 UNC institutions engaged in teacher education for their
valuable input during the development of the models and discussions of the findings. We
gratefully acknowledge the many contributions made by our current and former researchers and
fellows at the Education Policy Initiative at Carolina (EPIC), including Kevin C. Bastian, C.
Kevin Fortner, David C. Kershaw, Jade V. M. Jenkins, Charles L. Thompson, and Rebecca A.
Zulli. Finally, we wish to acknowledge the editing and formatting work done by Elizabeth
D’Amico, who is responsible for the overall look and polish of the report. All authors accept
responsibility for any remaining errors in the report.
Department of Public Policy Education Policy Initiative at Carolina
University of North Carolina at Chapel Hill
140 Friday Center Drive, Chapel Hill, NC 27599-9000
919-962-0668 publicpolicy.unc.edu
Introduction
This report, produced in collaboration with the UNC General Administration, presents the third1
set of results assessing the effectiveness of undergraduate teacher preparation programs at North
Carolina public universities. UNC established this research agenda in 2009 in order to quantify
the effects of UNC traditional (undergraduate) preparation programs on student achievement in
North Carolina and to provide results to assist UNC Schools of Education in the evaluation and
improvement of their programs. The purpose of this analysis is to accurately estimate the
average test score gains for public school students taught by recent graduates of each of UNC’s
fifteen undergraduate teacher preparation programs, net of other variables that affect student
performance. Thus, this report applies the value-added models used in prior studies, with
extensive student, classroom, and school covariates, in order to isolate the effects of traditional
UNC teacher preparation programs on student achievement in the public schools of North
Carolina.
Generating quantitative estimates of program effectiveness allows the UNC institutions to see
where their program graduates perform well, provides a starting point for inquiry about program
characteristics that affect student achievement scores, and identifies programs that need
improvement. The information in this report covers programmatic effects across the entire range
of grade levels and subjects tested by public schools in North Carolina. Estimates of program
effectiveness vary widely across these subject and grade level combinations, and therefore, there
is no single measure of overall effectiveness for each program.
Two types of models were employed for this analysis: 1) models comparing the graduates of
each of the 15 public undergraduate preparation programs to all other teachers in the state, and 2)
individual campus models comparing each undergraduate program to 12 other categories of
teacher preparation, including alternative entry and teachers prepared out-of-state. These two
approaches serve different purposes. The first allows comparisons between each of the
undergraduate teacher preparation programs and a common reference group representing the
average of all other types of teacher preparation. Essentially, this first analysis shows how
effective each of the undergraduate programs is by comparing it to the teaching corps that North
Carolina would have if the traditional UNC programs did not exist. This analysis is the focus of
this report. The second set of models provides a more detailed comparison between
undergraduate prepared teachers from an individual UNC institution and teachers prepared
through the other major means by which teachers are prepared to teach in North Carolina. This
set of value-added models specifies each UNC institution as the reference group and makes
direct comparisons between the reference institution and 12 other categories of teacher
preparation, including teachers from other in-state public institutions, in-state private institutions,
1 See Henry, G.T., Thompson, C.L., Fortner, C.K., Zulli, R.A., and Kershaw, D.C (2010). The Impact of Teacher
Preparation on Student Learning in North Carolina Public Schools. Chapel Hill, NC: The Carolina Institute for
Public Policy. Available online at:
http://publicpolicy.unc.edu/research/Teacher_Prep_Program_Impact_Final_Report_nc.pdf and
Henry, G.T., Thompson, C.L., Bastian, K.C., Fortner, C.K., Kershaw, D.C., Marcus, J.V., and Zulli R.A. (2011)
UNC Teacher Preparation Program Effectiveness Report. Chapel Hill, NC: The Carolina Institute for Public Policy.
Available online at: http://publicpolicy.unc.edu/research/TeacherPrepEffectRpt_Final.pdf
2
out-of-state universities, Teach For America, and other alternative entry teachers. The 12 other
categories of teachers are defined in the 2011 UNC Teacher Preparation Program Effectiveness
Technical Report.2 We provide results from this second set of models to the deans of each
institution in a separate institution-specific report.
These estimates of the effects of traditional UNC teacher preparation programs reflect both the
knowledge and skills that teachers develop while they are prospective teachers at a particular
university and the capacity or learned ability that the prospective teachers bring with them into a
particular preparation program. While it may be useful to separate these effects to determine
which programs add more value during the preparation process, the education that a teacher
provides to a student in North Carolina public schools is a product of both preparation and
academic ability. Therefore, these estimates are intended to capture the total effect of teacher
preparation and selection in a single estimate of value-added model outcomes.
The effectiveness of teacher preparation programs should be judged, at least in part, on the extent
to which teachers prepared by these programs are able to produce gains in students’ knowledge
and skills. To assess these gains we use the standards-based state assessments as the measures of
student learning. In this report, we estimate the effects of teachers on student performance on the
available End-of-Grade (EOG) and End-of-Course (EOC) tests administered by the North
Carolina Department of Public Instruction (NC DPI). These test scores have strengths and
weaknesses as a method for estimating the effectiveness of teachers. The student assessment
program in North Carolina serves as a model for other states and the state’s test scores are widely
used in rigorous, high quality research and evaluation studies. North Carolina’s state tests are
aligned with the NC Standard Course of Study within each grade and/or subject, and tests in
reading and mathematics in grades 3 to 8 are vertically scaled to allow for comparisons across
years and grades. Student EOG and EOC test scores serve as the basis for the ABCs
accountability program in North Carolina as well as for calculating the value-added by teachers
and schools to student test score growth which will be used in the state’s new system for
evaluating teachers and principals. Because EOG and EOC tests are closely aligned with the
intended curriculum, the test score gains produced by program graduates are appropriate for
estimating the effectiveness of teachers who graduate from UNC teacher preparation programs.
The weaknesses of using student test score outcomes to measure teaching effectiveness include
the limited subjects and grades in which testing is conducted—students below grade three are not
tested; students in 3rd
through 8th
grade are tested only in reading and mathematics, annually, and
in science in only the 5th
and 8th
grade; and the number of EOC tests in high school has decreased
in recent years. This prevents any estimates of teacher effectiveness based on test scores in other
subject or grade level combinations and means that program effectiveness estimates are based on
a subset of program graduates and not on all teachers prepared by the preparation program (see
Table 3). Other important outcomes, such as graduation, attitudes toward school and learning, or
capacity to function as a citizen within a democracy, are not captured by these standardized tests.
The tests measure the extent to which individual teachers succeed in teaching their students the
state-adopted curriculum in specific grades and subjects. So while limited, they do provide
useful outcome-based information for assessing the effectiveness of teacher preparation
programs. 2 Henry, G.T., Thompson, C.L., Fortner, C.K., Bastian, K.B. and., Marcus, J.V. (2011) Technical Report: UNC
Teacher Preparation Program Effectiveness Report. Available online at:
http://publicpolicy.unc.edu/research/ProgramEffect_TechRpt_July2011.pdf
3
This report provides an overview of the data and methodology of the current analysis and the
changes between this evaluation of teaching program effectiveness and the prior report.3
Subsequent sections include the results of the analysis—comparing each undergraduate program
to the aggregate of all other sources of teacher preparation—and a conclusion section. We
provide separate campus reports (not included here) to each institution with results from the
individual campus models.
Data and Methods
This 2013 edition of the UNC Teacher Preparation Program Effectiveness Report continues to
utilize a value-added modeling approach that compares the average student learning gains for
students whose teachers have different types of preparation prior to entering the classroom in
North Carolina public schools. These value-added models are estimated within a multi-level
model specification with extensive controls at the student, classroom, and school level. Limited
controls for teachers (years of experience and out-of-field teaching) are included in the models in
order to generate comparable estimates of the total effect of teacher preparation programs on
student achievement. For example, although we expect teachers with higher Praxis scores and
National Board Certification to generate greater learning gains for their students, on average, we
exclude these variables from the analysis since teachers from certain institutions may be more
likely to have some of these characteristics, and including them would downwardly bias program
effect estimates.
Table 1 includes a complete list of control variables included at each of the three levels in the
multi-level model. The definitions for three of the variables may not be obvious: structural
mobility refers to students who changed schools due to the grade configuration of a school (i.e.
6th grade students in a 6 - 8 middle school); between-year mobility refers to students who either
attended or completed assessments at a different school in the prior academic year; and within-
year mobility refers to students who were enrolled in the school where they took their tests for
substantially less than the full school year (more than two weeks less than a full year).
3 Henry, G.T., Thompson, C.L., Bastian, K.C., Fortner, C.K., Kershaw, D.C., Marcus, J.V., and Zulli R.A. (2011)
UNC Teacher Preparation Program Effectiveness Report. Chapel Hill, NC: The Carolina Institute for Public Policy.
Available online at: http://publicpolicy.unc.edu/research/TeacherPrepEffectRpt_Final.pdf
4
Table 1: Standard Model Control Variables
Student Classroom & Teacher School
1. Prior test scores (reading
& math)
2. Classmates prior test
scores (peer effects)
3. Days absent
4. Structural mobility
5. Between-year mobility
6. Within-year mobility
7. Gender
8. Race/ethnicity
9. Poverty
10. Gifted
11. Disabled
12. Currently limited English
proficient
13. Previously limited
English proficient
14. Overage for grade (held
back or retained at least
once)
15. Underage for grade
(promoted two grades)
16. Years of experience
17. Teaching out-of-field
18. Number of students
19. Advanced curriculum
20. Remedial curriculum
21. Dispersion of prior
achievement within
classroom
22. School size (ADM)
23. School size squared
24. Suspension rate
25. Violent acts rate
26. Total per pupil expenditures
27. District teacher supplements
28. Racial/ethnic composition
29. Concentration of poverty
Table 2 (below) lists the 15 campuses included in the program comparison results contained in
this report and the abbreviations for each campus used throughout the report. Separate reports
present the results of each of these campuses compared to 12 other categories of teacher
preparation. To identify graduates of traditional UNC preparation programs for this analysis, we
rely on data from the UNC General Administration, which includes information on the
undergraduate major(s) and campus enrollment for each individual who attended a North
Carolina four-year public institution. Individuals are classified as program graduates if they
graduated from a UNC institution with an education major or graduated with another major and
simultaneously received a teaching license. These individuals are considered traditionally
prepared teachers in this report regardless of any additional preparation (such as a Master’s
degree from another institution) obtained between graduating from the traditional teacher
education program and starting work as a classroom teacher. This definition allows the UNC
teacher preparation program comparisons to include all of the traditional undergraduate prepared
teachers that graduated from each institution, but creates slight differences between the teacher
categories specified in this report and the Education Policy Initiative at Carolina’s prior
publication on portal effectiveness.4
4 Portal Report: Teacher Preparation and Student Test Scores in North Carolina. (2010). Chapel Hill, NC: The
Education Policy Initiative at Carolina (EPIC) formally known as the Carolina Institute for Public Policy. Available
online at:
http://publicpolicy.unc.edu/research/Teacher_Portals_Teacher_Preparation_and_Student_Test_Scores_in_North_Ca
rolina_2.pdf
5
Table 2: UNC Institution Abbreviations
Teacher Preparation Program Abbreviation
Appalachian State University ASU
East Carolina University ECU
Elizabeth City State University ECSU
Fayetteville State University FSU
North Carolina Agricultural & Technical State University NCA&T5
North Carolina Central University NCCU
North Carolina State University NCSU
University of North Carolina – Asheville UNCA
University of North Carolina – Chapel Hill UNCCH
University of North Carolina – Charlotte UNCC
University of North Carolina – Greensboro UNCG
University of North Carolina – Pembroke UNCP
University of North Carolina – Wilmington UNCW
Western Carolina University WCU
Winston-Salem State University WSSU
The data used for this analysis includes student, classroom, and school characteristics from
public schools in North Carolina from the 2006-07 through the 2010-11 school years. This five-
year span updates the data used (2005-06 through 2009-10) in our previous analysis. In addition,
we restrict the models to teachers with less than five years of teaching experience. This allows us
to assess the effectiveness of relatively recent graduates from each program. The cutoff of less
than five years of experience balances the need for a sample of program graduates that is large
enough to calculate effects and recognizes that the effects of a teacher’s training program will
diminish over time as a teacher learns from classroom experience, principal and peer feedback,
and other professional development.
This report includes results from 11 distinct models for student test score outcomes:
elementary school mathematics, reading (grades 4 and 5), and science (grade 5 only); middle
school mathematics, reading (grades 6-8), science (grade 8 only), and algebra I; and high school
(grades 9 – 12) algebra I & II, English I, science (biology and physical science), and social
studies (US history and civics & economics). The previous report did not include analysis of
elementary school science. High school models are slightly different from the prior analysis due
to the elimination of a number of EOC tests (North Carolina discontinued the geometry,
chemistry, and physics tests during our study period). Finally, we eliminated the high school
overall model due to the changes in tests previously included in that analysis.
We present results in a graphical format that resembles a thermometer and displays institutions
or teacher preparation categories vertically, corresponding to their estimates of effectiveness.
These graphs provide an accessible visual presentation of the data, but introduce the need for
some explanatory notes. First, only comparisons with categories that contain at least ten teachers
5 Abbreviated as NCAT in Figures 1, 2, and 3
6
from a given teacher preparation program are reported. This count is based on unique teachers
across years and may include observations of student outcomes over numerous years that are
linked to a given teacher. Second, results which are statistically significant at the 0.05 level are
marked with asterisks and bolded to indicate they are significantly different from the reference
category, teachers from all sources other than undergraduate preparation programs at UNC
institutions. We should also note that these estimates are reported as a percentage of a standard
deviation unit, meaning for example, that a value of 25 indicates that the students taught by a
teacher from a specific program are expected to generate test score outcomes 25 percent of a
standard deviation higher than similar students in similar schools and classrooms taught by
teachers not prepared by undergraduate preparation programs at UNC institutions.
Table 3 presents the total number of teachers from each institution who were teaching in North
Carolina public schools during the five year study period, the number of teachers with less than 5
years of experience teaching in NC public schools over the time period, and the number of
teachers with less than 5 years of experience who were teaching in tested grades and subjects,
and therefore, could be included in the models. In cases of missing or unmatched data, teachers
are excluded from models. (See Table 5 for exact counts of teachers included in each model).
7
Table 3: UNC Institutional Counts
Teacher
Preparation
Program
Total Teachers
from
2006-07 to 2010-111
Total Teachers with Less than
5 Years’ Experience
from
2006-07 to 2010-112
Total Teachers in Tested Grades
and Subjects with Less than 5
Years’ Experience from
2006-07 to 2010-113
ASU 6850 3074 1530
ECU 6537 2828 1263
ECSU 716 238 101
FSU 1253 555 282
NCA&T 698 282 128
NCCU 862 356 167
NCSU 1852 1030 614
UNCA 381 211 135
UNCCH 1924 629 281
UNCC 3342 1783 940
UNCG 3956 2019 839
UNCP 1784 681 316
UNCW 3204 1630 808
WCU 2425 1053 514
WSSU 614 197 77 1Represents the total number of unique, traditionally prepared teachers paid as certified instructors in North
Carolina public schools at least one time between 2006-07 and 2010-11. Counts in this column are for all teachers,
regardless of experience or subject/grade taught. 2 Represents the total number of unique, traditionally prepared teachers with less than five years of experience paid
as certified instructors in North Carolina public schools at least one time between 2006-07 and 2010-11. Counts in
this column are for all teachers with less than five years of experience, regardless of subject/grade taught. 3Represents the total number of unique, traditionally prepared teachers with less than five years of experience paid
as certified instructors in North Carolina public schools at least one time between 2006-07 and 2010-11 who taught
in tested subjects and grade levels during that time period.
8
Results
Graduates of traditional teacher preparation programs at each UNC institution were compared to
all other sources of teachers on 11 different standardized tests (EOG or EOC), with three
outcomes in elementary school, four in middle school, and four in high school. Results are not
reported in cases where a program had fewer than ten graduates teaching the grade and subject
associated with the EOG or EOC test. For elementary and middle school mathematics and
reading we report results in terms of equivalent days of instruction gained (or lost) by
comparable students whose teacher graduated from a particular program compared to all other
sources of teachers. We cannot report days equivalency in elementary school science, middle
school science, middle school algebra I, nor any high school EOC tests because the tests are not
vertically equated to a prior test score in the same subject.
The Effects of UNC Teacher Preparation Programs on Student Achievement
Appalachian State University (ASU): On average, graduates of Appalachian State
University’s undergraduate teacher preparation program were more effective than all other
sources of teachers in one comparison – elementary school mathematics. It may be useful to
think of this impact in terms of days of learning. A student with a teacher prepared at ASU could
gain an average of 6 days of learning per year more than a student instructed by a teacher in the
reference category.6 ASU graduates performed similarly to all other sources of teachers in the
remaining ten comparisons – elementary school reading, elementary school science, middle
school mathematics, middle school reading, middle school science, middle school algebra I, high
school algebra I and algebra II, high school science, high school English I, and high school social
studies.
East Carolina University (ECU): On average, graduates of East Carolina University’s
undergraduate teacher preparation program were more effective than all other sources of teachers
in three comparisons – elementary school reading, middle school algebra I, and high school
social studies. In elementary school reading, a student with a teacher prepared at ECU could
gain an average of 5 ½ days of learning per year more than a student instructed by a teacher in
the reference category. ECU graduates performed similarly to all other sources of teachers in the
remaining eight comparisons – elementary school mathematics, elementary school science,
middle school mathematics, middle school reading, middle school science, high school algebra I
& II, high school English I, and high school science.
Elizabeth City State University (ECSU): On average, graduates of Elizabeth City State
University’s undergraduate teacher preparation program performed similarly to all other sources
of teachers in five comparisons – elementary school mathematics, elementary school reading,
elementary school science, middle school mathematics, and middle school reading. There were
insufficient (<10) ECSU graduates teaching middle school science, middle school algebra I, high
school algebra I and algebra II, high school science, high school English I, and high school social
studies to report results from these comparisons.
6 See Appendix for instructions on calculating days equivalency.
9
Fayetteville State University (FSU): On average, graduates of Fayetteville State
University’s undergraduate teacher preparation program were more effective than all other
sources of teachers in one comparison – high school algebra I & II. FSU graduates performed
similarly to all other sources of teachers in seven comparisons – elementary school mathematics,
elementary school reading, elementary school science, middle school mathematics, middle
school reading, high school English I, and high school social studies. There were insufficient
FSU graduates teaching middle school science, middle school algebra I, and high school science
to report results from these comparisons.
North Carolina A&T State University (NCA&T): On average, graduates of North
Carolina A&T State University’s undergraduate teacher preparation program performed
similarly to all other sources of teachers in five comparisons – elementary school mathematics,
elementary school reading, elementary school science, middle school mathematics, and high
school algebra I & II. On average, NCA&T graduates were less effective than all other sources
of teachers in one comparison – middle school reading. There were insufficient NCA&T
graduates teaching middle school science, middle school algebra I, high school English I, high
school science, and high school social studies to report results from these comparisons.
North Carolina Central University (NCCU): On average, graduates of North Carolina
Central University’s undergraduate teacher preparation program were more effective than
teachers from all other sources in one comparison – elementary school science. NCCU
graduates performed similarly to all other sources of teachers in four comparisons – elementary
school mathematics, elementary school reading, middle school mathematics, and middle school
reading. There were insufficient NCCU graduates teaching middle school science, middle
school algebra I, high school science, and high school social studies to report results from these
comparisons.
North Carolina State University (NCSU): On average, graduates of North Carolina
State University’s undergraduate teacher preparation program were more effective than all other
sources of teachers in one comparison – high school social studies. NCSU graduates performed
similarly to all other sources of teachers in the remaining ten comparisons – elementary school
mathematics, elementary school reading, elementary school science, middle school mathematics,
middle school reading, middle school science, middle school algebra I, high school algebra I and
algebra II, high school English I, and high school science.
University of North Carolina – Asheville (UNCA): On average, graduates of
University of North Carolina at Asheville’s undergraduate teacher preparation program
performed similarly to all other sources of teachers in seven comparisons – elementary school
mathematics, elementary school reading, elementary school science, middle school mathematics,
middle school reading, high school English I, and high school science. On average, UNCA
graduates were less effective than all other sources of teachers in one comparison – high school
algebra I & II. There were insufficient UNCA graduates teaching middle school science, middle
school algebra I, and high school social studies to report results from these comparisons.
University of North Carolina – Chapel Hill (UNCCH): On average, graduates of
University of North Carolina at Chapel Hill’s undergraduate teacher preparation program were
more effective than all other sources of teachers in one comparison – middle school
mathematics. In terms of days of learning, a student with a teacher prepared at UNCCH could
gain an average of 44 days of learning per year more than a student instructed by a teacher in the
10
reference category. UNCCH graduates performed similarly to all other sources of teachers in
five comparisons – elementary school mathematics, elementary school reading, elementary
school science, middle school reading, and middle school science. There were insufficient
UNCCH graduates teaching middle school algebra I, high school algebra I and algebra II, high
school science, high school English I, and high school social studies to report results from these
comparisons.
University of North Carolina – Charlotte (UNCC): On average, graduates of
University of North Carolina at Charlotte’s undergraduate teacher preparation program were
more effective than all other sources of teachers in two comparisons – high school science and
high school social studies. UNCC graduates performed similarly to all other sources of teachers
in eight comparisons – elementary school mathematics, elementary school reading, elementary
school science, middle school mathematics, middle school reading, middle school science, high
school algebra I & II, and high school English I. On average, UNCC traditionally prepared
teacher were less effective than all other sources of teachers in one comparison – middle school
algebra I.
University of North Carolina – Greensboro (UNCG): On average, graduates of
University of North Carolina at Greensboro’s undergraduate teacher preparation program were
more effective than all other sources of teachers in one comparison – elementary school
mathematics. A student with a teacher prepared at UNCG could gain an average of 7 ½ days of
learning per year more than a student instructed by a teacher in the reference category. UNCG
graduates performed similarly to all other sources of teachers in nine comparisons – elementary
school reading, elementary school science, middle school mathematics, middle school reading,
middle school algebra I, high school algebra I and algebra II, high school science, high school
English I, and high school social studies. There were insufficient UNCG graduates teaching
middle school science to report results from this comparison.
University of North Carolina – Pembroke (UNCP): On average, graduates of
University of North Carolina at Pembroke’s undergraduate teacher preparation program
performed similarly to all other sources of teachers in eight comparisons – elementary school
mathematics, elementary school reading, elementary school science, middle school mathematics,
middle school reading, high school algebra I & II, high school science, and high school social
studies. There were insufficient UNCP graduates teaching middle school science, middle school
algebra I, and high school English I to report results from these comparisons.
University of North Carolina - Wilmington (UNCW): On average, graduates of
University of North Carolina Wilmington’s undergraduate teacher preparation program were
more effective than all other sources of teachers in two comparisons – elementary school science
and middle school mathematics. A middle school mathematics student with a teacher prepared
at UNCW could gain an average of 32 ½ days of learning per year as compared to a student
instructed by a teacher in the reference category. UNCW graduates performed similarly to all
other sources of teachers in the remaining nine comparisons – elementary school mathematics,
elementary school reading, middle school reading, middle school science, middle school algebra
I, high school algebra I and algebra II, high school science, high school English I, and high
school social studies.
11
Western Carolina University (WCU): On average, graduates of Western Carolina
University’s undergraduate teacher preparation program were more effective than all other
sources of teachers in one comparison – high school English I. WCU graduates performed
similarly to all other sources of teachers in the remaining ten comparisons – elementary school
mathematics, elementary school reading, elementary school science, middle school mathematics,
middle school reading, middle school science, middle school algebra I, high school algebra I &
II, high school science, and high school social studies.
Winston-Salem State University (WSSU): On average, graduates of Winston-Salem
State University’s undergraduate teacher preparation program performed similarly to all other
sources of teachers in five comparisons – elementary school mathematics, elementary school
reading, elementary school science, middle school mathematics, and middle school reading.
There were insufficient WSSU graduates teaching middle school science, middle school algebra
I, high school algebra I and algebra II, high school science, high school English I, and high
school social studies.
The figures below present the results of models that compare the average effectiveness of
teachers from each of UNC’s fifteen traditional undergraduate teacher preparation programs to
the average effectiveness of teachers from all other sources of preparation. Where estimates are
statistically different from all other sources of teachers (significant at the 0.05 level), institution
names appear in bold text, are noted with a *, and have points designated with diamonds.
Institutions with insufficient data to report results (fewer than ten teachers in the model) are
excluded from graphs. The scaling of graphs may differ to maximize the readability of each
graph. The results are summarized in Table 4.
12
Figure 1: UNC Traditional Preparation Programs vs. All Other Sources of Teachers – Elementary School
13
Figure 2: UNC Traditional Preparation Programs vs. All Other Sources of Teachers – Middle School
14
Figure 3: UNC Traditional Preparation Programs vs. All Other Sources of Teachers – High School
15
Table 4: Summary of Results
The results summarized in Table 4 (above) are based on the number of statistically significant
positive or negative comparisons for each institution across the 11 models. Table 5 (below)
displays the number of teachers, by institution, included in each model. Higher numbers of
teachers in a particular group decrease the standard errors and make statistically significant
findings more likely. Schools with small sample sizes are unlikely to show statistically
significant results, even when the estimated effect is large in absolute value terms.
Institutions with
Two or More
Positive Results
Outperforming the
Reference Group Comparisons
Underperforming the
Reference Group Comparisons
ECU ES Reading; MS Algebra I;
HS Social Studies ----
UNCC HS Science; HS Social Studies MS Algebra I
UNCW ES Science; MS Mathematics ----
Institutions with
One Positive Result
Outperforming the
Reference Group Comparisons
Underperforming the
Reference Group Comparisons
ASU ES Mathematics ----
FSU HS Algebra I & II ----
NCCU ES Science ----
NCSU HS Social Studies ----
UNCCH MS Mathematics ----
UNCG ES Mathematics ----
WCU HS English I ----
Institutions with
No Statistically Significant
Results
Outperforming the
Reference Group Comparisons
Underperforming the
Reference Group Comparisons
ECSU ---- ----
UNCP ---- ----
WSSU ---- ----
Institutions with
One Negative Result and
No Positive Results
Outperforming the Reference
Group Comparisons
Underperforming the
Reference Group Comparisons
NCA&T ---- MS Reading
UNCA ---- HS Algebra I & II
16
Table 5: Teacher Counts by UNC System School, Level and Subject
Teacher
Preparation
Program
ES
Math
ES
Read
ES
Science
MS
Math
MS
Read
MS
Science
MS
Algebra
I
HS
Algebra
I & II
HS
English
HS
Science
HS
Social
Studies
ASU 666 666 183 177 180 30 33 116 112 35 147
ECU 620 617 192 139 140 30 23 81 70 40 79
ECSU 59 59 20 16 10 2 1 1 5 2 3
FSU 102 102 32 43 48 5 4 39 18 4 18
NCA&T 58 58 18 17 11 2 6 16 8 3 3
NCCU 88 88 27 23 17 2 8 12 11 3 8
NCSU 31 31 13 86 122 32 34 146 58 43 65
UNCA 46 46 15 10 17 2 4 14 19 13 7
UNCCH 164 164 43 40 41 12 9 8 5 8 1
UNCC 497 497 138 113 91 28 21 60 34 12 50
UNCG 406 405 114 77 105 8 13 39 69 14 42
UNCP 143 143 42 24 34 7 1 29 9 13 25
UNCW 416 416 113 96 81 18 21 59 58 12 29
WCU 215 215 61 68 70 10 11 38 30 18 37
WSSU 39 39 10 10 12 1 1 3 1 0 3 *Highlighted cells have fewer than ten teachers and therefore do not have any results reported.
17
Results in elementary and middle school mathematics and reading models may be interpreted in
terms of the equivalent days of instruction gained (or lost) by comparable students whose teacher
graduated from a particular program compared to all other sources of teachers. Table 6 contains
values for interpretation of effectiveness estimates (coefficients) depending on the model under
consideration. For example, comparable students in similar classrooms and schools are expected
to score as if they had attended 14 and one-third extra days of school when they are taught by a
teacher whose effectiveness estimate (coefficient) is five percent of a standard deviation higher
than the reference group in elementary school mathematics. These estimates vary based on the
subject and grade level, and the formula for calculating values based on different results is found
in the appendix of this report.
Table 6: Key for the Interpretation of Coefficients (Days Equivalency)
Result Values
(Coefficient/Effectiveness) ES Math
ES
Reading MS Math
MS
Reading
15.00 43.13 days 49.16 days 97.71 days 79.35 days
10.00 28.76 days 32.78 days 65.14 days 52.90 days
5.00 14.38 days 16.39 days 32.57 days 26.45 days
2.00 5.75 days 6.56 days 13.03 days 10.58 days
Note: These result values show days equivalency in relation to the reference group; negative result values
have negative days equivalency results. See the appendix section for directions on calculating the days
equivalency.
Conclusion
Of the fifteen traditional teacher education programs at UNC institutions, most are performing
about as well, or better than, all other sources of teachers in terms of their estimated impact on
student EOG and EOC achievement tests. Only two of the campuses had results with more
negative model outcomes than positive, ten campuses demonstrated more positive model
outcomes than negative results, and three campuses had no statistically significant results.
Overall, on average, UNC traditionally prepared teachers are likely to outperform teachers from
all other sources combined, and some programs perform substantially better than others in
specific subjects or grade levels. We present the estimates comparing each campus to each of the
twelve “portals” or categories of teacher preparation in separate campus-specific reports.
18
Appendix: Calculating Days Equivalency
Table A.1. Necessary Information for Days Equivalency Calculations
End of Grade Test Standard Deviation Average Yearly Gains
Elementary School Mathematics 9.373 5.867
Elementary School Reading 9.614 5.280
Middle School Mathematics 9.174 2.535
Middle School Reading 8.931 3.039
Days Equivalency Equation:
(((Result value/100) x Standard Deviation)/ (Avg. Yearly Gain))) x 180
Example for Elementary School Mathematics:
Step One
Result value from institutional comparison graph = 2.10
Standard Deviation (9.373) and Average Yearly Gains (5.867) from the table above
Step Two
Insert the result value into the days equivalency equation
(((2.10/100) x 9.373)/(5.867))) x 180 = 6.04 days of learning
Days Equivalency Note:
Days equivalency values can be calculated for elementary and middle grades mathematics and
reading tests because the tests are interval scaled and students have prior test scores for the
subject. We cannot calculate days equivalency in elementary grades science, middle grades
science and algebra, and in all high school tests (EOC) because these assessments are not
vertically equated to a prior test score in the same subject.
publicpolicy.unc.edu
top related