7hfkqlfdo 5hsruw 81& 7hdfkhu 3uhsdudwlrq ......july 2011 by gary t. henry, unc-chapel hill charles...

Carolina Institute for Public Policy

Technical Report:

UNC Teacher Preparation ProgramEffectiveness Report

July 2011

Technical Report:

UNC Teacher Preparation Program Effectiveness

July 2011

by

Gary T. Henry, UNC-Chapel Hill

Charles L. Thompson, East Carolina University

C. Kevin Fortner, UNC-Chapel Hill

Kevin C. Bastian, UNC-Chapel Hill

Jade V. Marcus, UNC-Chapel Hill

Acknowledgements

We wish to recognize Alisa Chapman, Alan Mabe and Keith Brown with the University of North

Carolina General Administration for their vital contributions in providing data and working as

partners throughout the research and communication processes.

We also wish to thank the deans and department heads from the colleges, schools and

departments of education at the 15 UNC institutions engaged in teacher education for their

valuable input during the development of the models and discussions of the findings. We

gratefully acknowledge the many contributions made by our current and former researchers and

fellows at the Carolina Institute for Public Policy, including Ashu Handa, Doug Lauen, Adrienne

Smith, and Kelly Purtell. Finally, we wish to acknowledge the editing and formatting work done

by Elizabeth D‟Amico, who is responsible for the overall look and polish of the report. All

authors accept responsibility for any remaining errors in the report.

Table of Contents

Introduction 1

Overall Study Design: Value Added Models 1

Equations for Estimating Teacher Impacts 2

Measures and Study Sample 4

a. Focal Variables: UNC Teacher Preparation Programs 4

b. Focal Variables: Individual Program Analysis 5

c. Merging Multiple Data Sets to Create the Study Data Base 5

d. Outcome Variables: Student Test Scores 7

Covariates: Influences on Student Test Scores Controlled in the Analysis 7

Prior Student Test Scores 7

Student and Family Characteristics 8

Classroom and Teacher Covariates 9

School Level Covariates 9

Summary 10

Tables:

Table 1: Teacher Preparation Categories 3

Table 2: Student Roster Counts 6

Table 3A: UNC Teacher Preparation Program Models – Elementary School 11

Table 3B: UNC Teacher Preparation Program Models – Middle School 13

Table 3C: UNC Teacher Preparation Program Models – High School 15

Table 4: Calculating Days Equivalency 19

Introduction

The purpose of this Technical Report is to describe the methods used in the UNC Teacher

Preparation Program Effectiveness report and the reports for each teacher preparation program

in the University of North Carolina‟s fifteen institutions, which were published in July 2011. The

overall objective of this research is to estimate the average effects of UNC prepared teachers on

students‟ test scores in North Carolina. We analyzed more than 2.6 million test scores of 1.7

million students in elementary, middle, and high schools across the state, and estimated value-

added models using data on more than 28,000 teachers with less than five years of experience. In

addition to the average overall estimates of the effectiveness of teachers prepared by UNC

institutions, we completed 15 separate sets of models using each teacher preparation program as

the reference group and compared the test score gains of students taught by their graduates to the

test score gains of the students of teachers with 12 other types of teacher preparation.

In the main report, UNC Teacher Preparation Program Effectiveness, we described the

objectives and findings of the initial models comparing all UNC teacher preparation programs to

a common reference group, commented briefly on our methods, and interpreted the results. In

this companion Technical Report, we provide further details of the methods used to produce the

study findings, including our design, sample, and modeling approach. Furthermore, we include a

discussion of the design changes between these reports and the previous report, The Impact of

Teacher Preparation on Student Learning in North Carolina Public Schools (April 2010), the

equations used to generate estimates for the primary report and the companion institutional

reports, and the data compiled for this study. This report concludes with complete tables

including coefficient estimates for our overall analyses. Separate supplementary reports contain

the results of individual institution analyses.

Overall Study Design: Valued Added Models

In this study, we use year-to-year multilevel, value added models with extensive controls as we

did in the prior study. These models are as accurate, fair, feasible to implement consistently, and

transparent for use across the grade levels and subjects as possible. While our basic modeling

approach remained unchanged from prior reports comparing the effectiveness of teachers from

various teacher preparation programs, we changed the cutoff for teachers‟ years of experience,

added two more years of data, and added control variables. The bulleted points below provide a

list of changes between the prior teacher preparation models and the current report. A more

detailed explanation of these changes is included in the primary report.

• Added two years of data: 2008-09 and 2009-10 to 2005-06 through 2007-08 • Limited analysis to teachers with less than five years of experience • Included other teacher preparation categories in the models

Technical Report: UNC Teacher Preparation Program Effectiveness Report

Page 2 of 19

• Individual campus models compare each campus to other sources of teacher preparation • Included middle grade science and algebra 1 models • Presented findings in a graphical format • Expressed effects as percentages of a standard deviation

In addition, for the subjects of Elementary Reading and Mathematics and Middle School

Reading and Mathematics, we calculate an equivalent days of schooling value to provide a

meaningful metric to understand the effects of having a teacher from a specific preparation

program compared to a teacher from another program. Table 4 provides the formulas for this

calculation.

Equations for Estimating Teacher Impacts

In our first model, we estimate a separate effect for each of UNC‟s 15 institutions that prepare K-

12 teachers. Separate indicator variables for teachers prepared by each of the UNC institutions

generate specific estimates of the average teacher value added for students‟ of UNC prepared

teachers, compared to all other sources of teacher preparation in the North Carolina Public

Schools. Only teachers with less than five years of experience are included in the models, and

additional teacher characteristics variables separate the effects of years of experience and out-of-

field teaching from program effectiveness estimates. Traditional undergraduate teacher training

includes students who majored in a teacher preparation degree program, and those students

majoring in another subject who take classes in a school of education to concurrently gain a

teaching credential.

The equation used to estimate the effect of programs is:

Where is a student‟s test score, specifically student i in classroom j in school s in the

current time period;

estimates the average effect on student test scores of the teachers from each of the 15

UNC traditionally prepared programs;

are indicator variables that equal 1 if the teacher is a traditionally prepared

undergraduate from the specified institution in the particular analysis and 0 if not;

represents a prior test score or scores;

represents a set of individual student characteristics for student i in classroom j in school s;


Page 3 of 19

represents a set of classroom characteristics in classroom j in school s;

represents a set of school characteristics in school s;

and and are disturbance terms representing unexplained variation at the individual,

classroom, and school levels, respectively.

The second set of models generates up to 11 individual model results (based on the student test

score outcome) for each of UNC‟s fifteen teacher preparation programs. Within each of these

models, indicator variables designate groups of teachers based on their preparation for entry into

the classroom and are designated by 12 separate variables listed in Table 1 below.

Table 1: Teacher Preparation Categories

Teacher Preparation Categories Abbreviation

Other UNC Traditionally Prepared Undergraduates OtherUNC

UNC Graduate Degree Prepared UNCGD

NC Private University Undergraduate Degree Prepared NCPUGD

NC Private University Graduate Degree Prepared NCPGD

Out-of-State University Undergraduate Degree Prepared OSUGD

Out-of-State University Graduate Degree Prepared OSGD

UNC Licensure Only UNCLO

Other Licensure Only OtherLO

Teach For America TFA

Visiting International Faculty VIF

Alternative Entry AE

Unclassifiable Unclass

Other UNC Traditionally Prepared Undergraduates OtherUNC

UNC Graduate Degree Prepared UNCGD


Page 4 of 19

Each of the 12 types of teacher preparation are compared to the teachers traditionally prepared

by a specific UNC institution, which is the reference group for that institution‟s analyses. Their

teacher value added estimate, as the reference group, is set to zero and the coefficients on each of

the 12 categories of teacher preparation indicate the average difference in student test score

outcomes for students‟ of teachers in the specified category as compared to the reference

institution. The “Other UNC Traditionally Prepared Undergraduates” group includes all

traditionally prepared UNC teachers not from the reference institution and is the only group

whose members vary across each of the analyses. The remaining categories are consistent in

membership, but their estimated coefficients vary based on the reference group‟s performance in

each model. The equation for individual institution comparison models is as follows:

Where is a student‟s test score, specifically student i in classroom j in school s at time t;

provide estimates of the average effect of the teachers from each of the 12 comparison

categories on student test score outcomes;

are indicator variables that equal 1 if the teacher is a member of the

category in the particular analysis and 0 if not;

represents a prior test score or scores;

represents a set of individual student characteristics for student i in classroom j in school s;

represents a set of classroom characteristics in classroom j in school s;

represents a set of school characteristics in school s;

and and are disturbance terms representing unexplained variation at the individual,

classroom, and school levels, respectively.

Measures and Study Sample

In this section we describe the way each of the measures were constructed and the sources of the

data that were used. In addition, this section provides more detailed information on the methods

used to construct the data sets included in the models.

a. Focal Variables: UNC Teacher Preparation Programs

UNC Teacher Preparation Program variables are created based on data provided by the

UNC General Administration and matched to value-added model data based on each


Page 5 of 19

teacher‟s identification number. More detail on teacher program variables is provided in

UNC Teacher Preparation Program Effectiveness.

b. Focal Variables: Individual Program Analysis

Teachers that are coded as graduates of UNC Teacher Preparation Programs become the

reference group in each individual program analysis. The focal variables in the analysis

are 12 comparison groups representing the various teacher preparation categories in

Table 1. These categories are identified using data from the UNC General

Administration, the North Carolina Department of Public Instruction (NCDPI), and

Teach for America. With one exception, teacher assignments to preparation categories

use similar rules for the assignment to portals utilized in Teacher Portals: Teacher

Preparation and Student Test Scores in North Carolina (June 2010). In the portals

analysis, an individual who acquires a master‟s degree after completing a traditional

undergraduate program, but before beginning teaching, is identified as being in a

graduate portal. In this analysis, persons retain their identity as a traditionally trained

graduate of a UNC program if they meet the criteria of a program, regardless of

additional preparation before entering the teaching profession.

c. Merging Multiple Data Sets to Create the Study Data Base

The data used in this study includes longitudinal student achievement scores linked to

class rosters, teacher characteristics including experience and out-of-field teaching status,

classroom characteristics, and school characteristics provided to researchers at the

Carolina Institute for Public Policy. Roster files that NCDPI receives from schools

provide information which allow us to link students to teachers. Student roster files

contain information on a student‟s classroom assignments and the name of the teacher

instructing each class. The student‟s name, date of birth, and a study-generated

identification number are used to match the student to test scores that occur in the current

and prior year. Student roster information is matched with current year test scores for

over 95% of the students listed on rosters. Approximately 90% of these students are

matched to their prior year test scores. Table 2 (below) displays the count of students in

the data sets by year. We match approximately 93% of teachers in the roster data to other

teacher data.


Page 6 of 19

Table 2: Student Roster Counts

School Level/Subject

Unique Students

Academic Year

2005-06

Unique Students

Academic Year

2006-07

Unique Students

Academic Year

2007-08

Unique Students

Academic Year

2008-09

Unique Students

Academic Year

2009-10

ES Reading 182,973 170,567 324,656 222,313 156,641

ES Mathematics 124,619 124,224 235,715 164,057 104,474

MS Reading 65,509 65,270 64,375 76,212 71,336

MS Mathematics 79,358 75,319 70,623 81,469 72,733

MS Science - - 23,529 30,884 28,078

MS Algebra I 4,614 4,903 4,560 5,459 5,075

HS All Subjects* 149,850 110,348 152,732 199,377 176,157

HS English I 27,273 26,562 26,133 34,189 28,990

HS Math 41,532 41,339 43,151 54,591 57,430

HS Science 40,007 - 38,648 53,919 39,039

HS Social Studies 41,040 42,446 44,801 56,678 50,698

* Observations may not sum to total for all subjects due to rounding of partial year observations.


Page 7 of 19

d. Outcome Variables: Student Test Scores

Eleven separate models are conducted when at least 10 teachers are included in the

reference group. Two elementary school (grades 3-5) models base outcomes on End-of-

Grade (EOG) tests in reading and mathematics. Four middle school grades (grades 6-8)

models base outcomes on EOG tests in reading, mathematics, 8th

grade science

(beginning with the 2008-09 school year), and End-of-Course (EOC) tests in Algebra I.

Algebra I models include limited numbers of students enrolling in Algebra I during

middle school grades. Five high school (grades 9-12) models base outcomes on tests in

all EOC tested subjects combined, English I only, mathematics EOC tests (Algebra I,

Algebra II, and Geometry), science EOC tests (Biology, Chemistry, Physical Science,

and Physics), and social studies EOC tests (Civics/Economics and US History). Not all

tests are conducted or reported in all years and the availability of data varies across

subjects and grade level. All EOG test scores are standardized within year, grade, and

subject (z scores). EOC test scores are standardized within year and subject, but not by

grade since students in various grades take the same test. Standardization of these

variables converts scaled scores into a format where the average score is zero and the

standard deviation of scores is one. A student with a standardized score of zero scored at

the mean or average when compared to all students in his/her same grade in the year the

student took the test. Therefore, a student with a score of zero in two consecutive years

increased his/her performance in the subject in the second year by the same amount as the

average increase in the state for his/her grade.

Covariates: Influences on Student Test Scores Controlled in the Analysis

In this section, we describe the variables, referred to as covariates, which may influence student

test score outcomes. We use these covariates to adjust or control for differences in the students

that graduate from different teacher preparation programs and where they choose to teach. These

factors are beyond the control of the program and therefore, their effects on student achievement

should be removed to better isolate programmatic effects. Most of the covariates are the same for

the elementary, middle and high school analyses, however, a few differences exist.

Prior Student Test Scores. For high school students, prior test scores are the student‟s 8th

grade

test scores in both reading and math. These scores are not necessarily from the prior year as

students in higher level grades (10th

– 12th

grades) took their 8th

grade EOG exams in more

distant prior years. For middle school students, we use the student‟s prior test performance on

EOG exams in both reading and math in the preceding year. For elementary school students, the

student‟s prior year EOG or 3rd

grade pretest (beginning of the year) scores are used. For these

pretest scores, when two scores are available they were averaged, and when only one score was

available (for example, 3rd

grade pretest scores in years when both tests were not administered) a

single score was used.


Page 8 of 19

Student and Family Characteristics. In addition to prior student test performance, the models

also include student level variables that adjust for any performance differences that may be due

to other factors measured at the student level. The prior test performance of the other students

within a class identifies the contribution of peers to a student‟s own achievement. This is a single

value consisting of the mean of prior test scores of other students within a class, excluding the

individual student. The student‟s number of absences during the school year is included as a

continuous variable.

A series of variables indicate student mobility. One variable is related to structural mobility, a

student moving to a new school because the current grade configuration of a school required

moving (i.e. a 9th

grade student enrolled in a grades 9-12 high school). Between-year mobility

indicates students who tested (elementary and middle school) or attended a different school (high

school) in the prior year and the school did not require a structural move. Students who could not

be matched to roster data from the immediate prior year are coded as missing for between-year

mobility. Within year mobility is based on the reported days enrollment for a student. If the

student‟s days enrolled value is more than 10 days less than the modal value for the school, the

student is coded as being a within-year mover.

Dichotomous indicator variables are used in models to indicate male, Asian, black, Hispanic,

multiracial, and American Indian students (the ethnicity categories designated by the state of

North Carolina). As a proxy for a student‟s family income level, we use three dichotomous

variables indicating free lunch eligible students, those eligible for reduced price lunch, and

finally those whose lunch payment status is missing. Student exceptionality codes are used to

generate an indicator variable if the student is gifted and another indicator variable if the student

has a disability. Students are coded as having a disability if their school district has recorded an

exceptionality code which includes behavioral, physical, learning or cognitive disabilities. Two

covariates related to each student‟s English proficiency are included. One variable indicates

whether a student currently receives services for Limited English Proficiency (LEP), and a

second variable indicates whether a student formerly received LEP services but does not

currently receive the services.

In order to control for student‟s prior school progress, we created dichotomous variables for

students who were overage for their grade, indicating they had likely been held back for a grade,

or underage, indicating they had likely skipped a grade. The variables were created based on the

combination of a student‟s date of birth and grade level. Based on the enrollment cutoff date of

October 16th

, we mark students as underage if their birth date occurs before the cutoff date for

their grade level and overage if a student‟s birth date occurs after the cutoff date one year later.

Finally, dichotomous variables representing the student‟s grade level are included in high school

analysis models.

In addition to these characteristics, high school models also include indicator variables to control

for the average score differences between different EOC tests. During the time period covered by


Page 9 of 19

this study, ten tests were required and included in the analysis: Algebra I, Algebra II, Geometry,

Biology, Chemistry, Physical Science, Physics, English I, Civics/Economics, and US History.

Algebra I serves as the reference group for analyses in models that include all subjects and

mathematics only models. Physics is the reference group in models that include only science

courses. US History serves as the reference group in models of high school social studies

achievement. Models of achievement in English I do not include any additional test specific

control variables since it is the only test subject in the category.

Classroom and Teacher Covariates. Four control variables related to classroom characteristics

are included in the models. The number of students in a classroom is included to control for

differences in achievement that may be related to class size. Also, we calculate peer dispersion, a

classroom level variable indicating the range of students‟ prior test scores within a classroom.

This variable is calculated based on the standard deviation of prior year test scores for each

student within a classroom, and is included to control for any potential effect of the range of

prior abilities of the students within a classroom on a teacher‟s instruction and students‟

achievement. In the high school and middle school analyses, two indicator variables are used to

indicate classrooms with an advanced or remedial curriculum. Classrooms are labeled as an

advanced curriculum classroom when their titles include the keywords „advanced‟, „honors‟, or

„AP‟. Remedial classroom labels are based on course titles including „remedial‟ or „resource‟.

Teacher controls include indicator variables for a teacher‟s years of teaching experience and an

additional variable indicating teachers who are teaching a class that is „out-of-field‟. Teaching

out-of-field occurs when a teacher is the teacher of record for a course where the teacher does

not hold an initial or continuing license in the specified certification area. The models for this

analysis are limited to teachers with less than five years of teaching experience. A series of

indicator variables designate teachers in their first, second, third, and fourth years of teaching,

with fifth year teachers as the reference category.

School Level Covariates. School level covariates include measures of school size and context.

Variables indicating school size, measured by the average daily membership of a school, per

pupil expenditures measured in hundreds of dollars, and the average local supplement paid to

teachers in the school measured in hundreds of dollars, are included in models. The suspension

rate indicates the suspension rate per 100 students. The violent acts rate indicates the number or

reported violent acts within a school per 1000 students. Models also include the concentration of

students within a school by ethnicity and free or reduced lunch status. The variables report the

percentage of students who are black, Hispanic, Asian, American Indian, and multiracial (the

ethnicity categories designated by the state of North Carolina).


Page 10 of 19

Summary

In this study, we use multilevel models with extensive covariates or control variables to isolate

the effects of teachers‟ preparation on the achievement of their students. We included two types

of analyses. The first compares the UNC‟s fifteen teacher preparation programs to the outcomes

of students taught by teachers from all other sources of teacher preparation. In the second

analysis, we compare each of UNC‟s teacher preparation programs individually, as the reference

group, to twelve other categories of teacher preparation. The findings from these individual

analyses are found in the teacher preparation program reports for each institution. Differences

attributed to preparation programs or preparation categories control for students‟ prior test scores

in all cases, which is often referred to as value-added modeling. We also include covariates to

adjust or control for characteristics of students and their families, classrooms, and school context.

We estimate the effects of the graduates of UNC teacher preparation programs compared to the

effects of all other teachers, including private institutions, those prepared out of state, and those

alternative or lateral entry teachers that began teaching prior to obtaining a license from the state.


Page 11 of 19

Table 3A: UNC Teacher Preparation Program Models – Elementary School

Variable

Elementary Reading Elementary Mathematics

Value

Standard

Error Value

Standard

Error

Teacher Preparation Categories

ASU -0.000 0.006 0.021* 0.007

ECU 0.019* 0.007 0.004 0.009

ECSU -0.012 0.019 -0.044 0.030

FSU -0.011 0.015 0.027 0.019

NCA&T 0.024 0.022 0.030 0.031

NCCU 0.001 0.015 -0.032 0.022

NCSU -0.029 0.038 0.104 0.074

UNCA -0.018 0.020 0.039 0.022

UNCCH 0.008 0.012 0.010 0.013

UNCC -0.006 0.006 -0.011 0.008

UNCG 0.008 0.007 0.030* 0.010

UNCP 0.027 0.016 0.033 0.023

UNCW 0.019* 0.007 0.014 0.011

WCU 0.018 0.011 -0.001 0.014

WSSU -0.009 0.026 -0.017 0.030

Teacher Characteristics

Out of field teaching 0.007 0.005 -0.009 0.006

First year teacher -0.046* 0.004 -0.088* 0.006

Second year teacher -0.025* 0.004 -0.030* 0.006

Third year teacher -0.017* 0.004 -0.007 0.006

Fourth year teacher -0.008 0.004 -0.007 0.006

Student Characteristics

Average prior grade EOG scores (Std.) 0.678* 0.002 0.706* 0.001

Average peer test score (Prior grade) 0.043* 0.005 -0.001 0.006

Days absent -0.003* 0.000 -0.006* 0.000

Student structural move -0.050* 0.012 -0.042* 0.019

Student moved in prior year 0.005 0.004 0.002 0.003

Missing prior year school information 0.021* 0.005 0.044* 0.006

Student moved during school year -0.053* 0.005 -0.074* 0.005

Missing within year moved information --- --- --- ---

Underage student based on grade 0.053* 0.010 0.076* 0.009

Overage student based on grade -0.099* 0.003 -0.120* 0.002

Academically or intellectually gifted 0.255* 0.003 0.245* 0.003

Disabled student -0.132* 0.003 -0.117* 0.003

Free lunch -0.097* 0.003 -0.064* 0.002

Reduced price lunch -0.063* 0.004 -0.041* 0.004

Lunch status missing -0.095* 0.009 -0.021* 0.007


Page 12 of 19

Table 3A: UNC Teacher Preparation Program Models – Elementary School Continued

Variable

Elementary Reading Elementary Mathematics

Value

Standard

Error Value

Standard

Error

Black -0.170* 0.003 -0.145* 0.003

Hispanic -0.032* 0.005 -0.009* 0.005

Multiracial -0.053* 0.005 -0.043* 0.004

American Indian -0.084* 0.011 -0.060* 0.011

Asian 0.002 0.006 0.083* 0.006

Male -0.012* 0.002 0.020* 0.002

LEP services recipient -0.160* 0.006 -0.073* 0.005

Previous LEP services recipient 0.030* 0.007 0.051* 0.006

Classroom Characteristics

Students per classroom -0.001 0.000 -0.001 0.001

Classroom ability dispersion 0.046* 0.009 0.043* 0.012

School Characteristics

School size (per 100) -0.014* 0.004 -0.012* 0.005

School size squared 0.001* 0.000 0.001* 0.000

Total per-pupil expenditures ($100s) 0.000 0.000 0.000 0.000

Average teacher supplement ($100s) 0.000 0.000 0.001* 0.000

Short-term suspension rate (per 100 students) -0.001* 0.000 -0.001* 0.000

Violent acts rate (per 1000 students) 0.000 0.001 -0.002* 0.001

Free and reduced price lunch mean -0.001* 0.000 -0.001* 0.000

Black mean 0.000* 0.000 -0.000 0.000

Hispanic mean 0.001* 0.000 0.000 0.000

Multiracial mean -0.001 0.001 -0.004* 0.001

American Indian mean -0.000 0.000 -0.001* 0.000

Asian mean 0.002* 0.001 0.003* 0.001 Note: Teachers with less than 5 years of experience and teaching an elementary school reading or math course during the

2005-06, through 2009-10 school year.

*Indicates a given coefficient is significant at the .05 level.


Page 13 of 19

Table 3B: UNC Teacher Preparation Program Models – Middle School

Variable

Middle Reading Middle Mathematics Middle Science Middle Algebra I

Value

Standard

Error Value

Standard

Error Value

Standard

Error Value

Standard

Error


ASU -0.010 0.008 -0.023* 0.012 0.067 0.051 -0.117 0.070

ECU 0.004 0.008 -0.021 0.014 -0.015 0.037 0.115 0.061

ECSU 0.016 0.021 -0.045 0.041 --- --- NR NR

FSU 0.005 0.013 0.011 0.021 NR NR NR NR

NC A&T -0.059 0.034 0.008 0.043 NR NR NR NR

NCCU 0.005 0.020 -0.027 0.025 NR NR NR NR

NCSU -0.009 0.008 -0.019 0.014 0.033 0.055 -0.019 0.050

UNCA -0.042* 0.021 0.043 0.030 NR NR NR NR

UNCCH -0.001 0.012 0.069* 0.021 0.069 0.056 NR NR

UNCC 0.017 0.010 0.009 0.012 0.036 0.047 -0.140* 0.069

UNCG -0.006 0.011 -0.021 0.020 NR NR NR NR

UNCP 0.014 0.017 0.069* 0.031 NR NR NR NR

UNCW 0.018 0.010 0.042* 0.015 0.004 0.049 0.051 0.098

WCU 0.003 0.015 -0.011 0.016 0.009 0.063 NR NR

WSSU -0.047 0.028 -0.032 0.066 NR NR NR NR


Out of field teaching -0.003 0.004 -0.018* 0.006 0.011 0.020 -0.088* 0.035

First year teacher -0.029* 0.005 -0.073* 0.008 -0.070* 0.026 -0.136* 0.044

Second year teacher -0.015* 0.005 -0.021* 0.008 -0.028 0.024 -0.083 0.043

Third year teacher -0.003 0.005 -0.001 0.008 0.031 0.023 -0.007 0.045

Fourth year teacher -0.000 0.005 0.001 0.008 -0.025 0.022 -0.007 0.042


Prior grade EOG reading score (Std.) 0.577* 0.002 0.140* 0.001 0.443* 0.004 0.157* 0.007

Prior grade EOG math score (Std.) 0.176* 0.002 0.582* 0.002 0.261* 0.004 0.649* 0.009

Average peer reading test score

(Prior grade) 0.072* 0.003 0.111* 0.004 0.034* 0.007 0.086* 0.024

Days absent -0.003* 0.000 -0.006* 0.000 -0.005* 0.000 -0.012* 0.001

Student structural move -0.018* 0.004 -0.031* 0.006 0.093 0.050 0.155 0.098

Student moved in prior year 0.013* 0.004 0.005 0.004 0.023* 0.009 -0.011 0.019

Missing prior year school info. -0.006 0.012 0.003 0.012 0.045* 0.017 0.050 0.043

Student moved during school year -0.041* 0.005 -0.079* 0.005 -0.065* 0.011 -0.123* 0.031

Missing within year moved info. -0.046 0.078 -0.120 0.100 -0.243 0.132 -0.893* 0.301


Page 14 of 19

Table 3B: UNC Teacher Preparation Program Models – Middle School Continued

Variable

Middle Reading Middle Mathematics Middle Science Middle Algebra I

Value

Standard

Error Value

Standard

Error Value

Standard

Error Value

Standard

Error

Underage student based on grade 0.040* 0.008 0.049* 0.008 0.100* 0.016 0.094* 0.025

Overage student based on grade -0.059* 0.002 -0.068* 0.002 -0.074* 0.005 -0.092* 0.012

Academically or intellectually gifted 0.118* 0.004 0.149* 0.004 0.144* 0.007 0.130* 0.009

Disabled student -0.110* 0.004 -0.054* 0.004 0.005 0.009 0.004 0.026

Free lunch -0.058* 0.002 -0.023* 0.002 -0.050* 0.005 0.015 0.012

Reduced price lunch -0.039* 0.004 -0.016* 0.003 -0.010 0.008 -0.024 0.016

Lunch status missing -0.037* 0.010 -0.017 0.009 -0.035 0.019 -0.031 0.036

Black -0.091* 0.003 -0.085* 0.003 -0.251* 0.006 -0.081* 0.012

Hispanic -0.009* 0.004 -0.002 0.004 -0.089* 0.009 -0.005 0.016

Multiracial -0.002 0.005 -0.026* 0.005 -0.066* 0.012 -0.004 0.022

American Indian -0.064* 0.010 -0.045* 0.008 -0.184* 0.020 0.016 0.050

Asian -0.020* 0.006 0.099* 0.006 -0.021 0.015 0.127* 0.016

Male -0.037* 0.002 0.010* 0.002 0.189* 0.004 -0.018* 0.007

LEP services recipient -0.126* 0.006 -0.001 0.005 -0.093* 0.012 0.086* 0.025

Previous LEP services recipient 0.011 0.008 0.054* 0.007 -0.004 0.015 0.078* 0.028


Students per classroom 0.000 0.000 -0.001 0.000 -0.001 0.001 0.000 0.002

Reading classroom ability dispersion 0.017* 0.007 -0.008 0.010 -0.007 0.016 0.141* 0.055

Advanced curriculum 0.014* 0.005 0.019* 0.006 0.127* 0.040 -0.051 0.028

Remedial curriculum -0.026* 0.007 -0.036* 0.010 0.007 0.059 0.364* 0.101


School size (per 100) -0.004 0.003 0.008 0.005 0.004 0.011 0.021 0.034

School size squared 0.000 0.000 -0.001* 0.000 0.000 0.001 0.001 0.002

Total per-pupil expenditures ($100s) -0.000 0.000 -0.000 0.000 -0.000 0.000 0.007* 0.002

Average teacher supplement ($100s) 0.000 0.000 0.000 0.000 0.003* 0.001 -0.004 0.002

Short-term suspension rate (per 100

students) -0.000* 0.000 -0.000* 0.000 -0.001* 0.000 0.000 0.001

Violent acts rate (per 1000 students) -0.000 0.000 -0.000 0.000 0.000 0.000 -0.004* 0.002

Free and reduced price lunch mean -0.000* 0.000 -0.000 0.000 -0.000 0.001 0.003 0.002

Black mean 0.001* 0.000 0.000 0.000 -0.001* 0.000 -0.007* 0.002

Hispanic mean 0.001* 0.000 0.000 0.000 -0.002* 0.001 -0.004 0.003

Multiracial mean 0.002 0.001 -0.003 0.002 0.006 0.004 0.051* 0.012

American Indian mean -0.001 0.000 -0.001 0.001 -0.002 0.001 -0.000 0.005

Asian mean 0.001 0.001 0.005* 0.001 0.002 0.002 0.012* 0.006

Note: Teachers with less than 5 years of experience and teaching a middle school EOG or EOC tested course during the 2005-06

through 2009-10 school year.

*Indicates a given coefficient is significant at the .05 level


Page 15 of 19

Table 3C: UNC Teacher Preparation Program Models – High School

Variable

High School

All Subjects

High School

English I

High School

Mathematics

High School

Science

High School

Social Studies

Value

Standard

Error Value

Standard

Error Value

Standard

Error Value

Standard

Error Value

Standard

Error


ASU 0.018 0.010 -0.014 0.011 0.018 0.018 0.075 0.039 0.002 0.016

ECU -0.012 0.013 -0.012 0.011 -0.010 0.022 -0.083* 0.040 0.066* 0.025

ECSU -0.117* 0.046 NR NR NR NR NR NR NR NR

FSU -0.044* 0.020 0.008 0.018 0.067* 0.029 NR NR -0.042 0.046

NC A&T -0.030 0.025 NR NR -0.030 0.032 NR NR NR NR

NCCU -0.043 0.032 0.012 0.033 -0.040 0.040 NR NR NR NR

NCSU 0.024* 0.009 0.012 0.010 0.022 0.014 -0.012 0.031 0.091* 0.021

UNCA 0.034 0.025 -0.027 0.022 0.010 0.076 0.043 0.091 NR NR

UNCCH -0.046 0.027 NR NR NR NR NR NR NR NR

UNCC 0.059* 0.012 0.028 0.017 0.048* 0.022 0.237* 0.083 0.067* 0.019

UNCG 0.008 0.013 -0.016 0.012 0.051 0.030 0.014 0.038 -0.003 0.026

UNCP -0.032 0.024 NR NR 0.002 0.046 0.078 0.061 -0.088 0.046

UNCW 0.050* 0.013 0.022 0.012 0.028 0.026 0.119 0.069 0.004 0.026

WCU 0.014 0.021 0.042* 0.019 -0.052 0.032 -0.130* 0.053 0.058 0.053

WSSU NR NR --- --- NR NR NR NR NR NR


Out of field teaching -0.031* 0.005 -0.002 0.006 -0.022* 0.009 -0.069* 0.015 -0.025* 0.012

First year teacher -0.095* 0.008 -0.026* 0.007 -0.092* 0.014 -0.090* 0.022 -0.126* 0.015

Second year teacher -0.029* 0.008 -0.011 0.008 -0.037* 0.014 0.003 0.022 -0.044* 0.015

Third year teacher -0.012 0.008 -0.002 0.007 -0.032* 0.015 0.016 0.023 -0.018 0.015

Fourth year teacher -0.002 0.008 -0.005 0.008 0.003 0.015 0.034 0.023 -0.013 0.016


Page 16 of 19

Table 3C: UNC Teacher Preparation Program Models – High School Continued

Variable

High School

All Subjects

High School

English I

High School

Mathematics

High School

Science

High School

Social Studies

Value

Standard

Error Value

Standard

Error Value

Standard

Error Value

Standard

Error Value

Standard

Error


8th Grade Math EOG score (Std.) 0.327* 0.002 0.189* 0.003 0.512* 0.004 0.370* 0.004 0.203* 0.003

8th Grade Reading EOG score (Std.) 0.301* 0.002 0.484* 0.003 0.097* 0.002 0.281* 0.003 0.417* 0.003

Average peer test score (Prior grade) 0.158* 0.005 0.104* 0.007 0.152* 0.009 0.103* 0.012 0.170* 0.008

Days absent -0.008* 0.000 -0.005* 0.000 -0.009* 0.000 -0.009* 0.000 -0.008* 0.000

Student structural move 0.067* 0.005 -0.040* 0.009 0.084* 0.006 0.008 0.009 -0.036* 0.014

Student moved in prior year 0.012* 0.004 -0.010 0.022 0.012 0.008 0.014 0.008 0.015* 0.006

Missing prior year school info. 0.046* 0.006 0.026 0.034 0.011 0.009 0.044* 0.009 0.073* 0.008

Student moved during school year -0.077* 0.004 -0.063* 0.007 -0.077* 0.007 -0.101* 0.010 -0.069* 0.008

Missing within year moved info. -0.010 0.054 -0.078 0.083 0.003 0.090 0.073 0.131 -0.227* 0.110

Underage student based on grade 0.108* 0.006 0.058* 0.011 0.082* 0.010 0.167* 0.013 0.132* 0.010

Overage student based on grade -0.107* 0.002 -0.106* 0.004 -0.104* 0.003 -0.119* 0.005 -0.105* 0.004

Academically or intellectually gifted 0.143* 0.003 0.167* 0.005 0.153* 0.005 0.140* 0.006 0.144* 0.005

Disabled student -0.050* 0.004 -0.170* 0.006 -0.053* 0.005 -0.030* 0.007 0.001 0.006

Free lunch -0.008* 0.002 -0.046* 0.004 0.009* 0.003 0.002 0.004 -0.016* 0.003

Reduced price lunch -0.004 0.003 -0.043* 0.005 0.009* 0.005 0.013* 0.006 -0.012* 0.005

Lunch status missing 0.003 0.006 -0.018 0.010 -0.002 0.009 0.001 0.011 0.015 0.010

Black -0.115* 0.003 -0.078* 0.005 -0.104* 0.004 -0.155* 0.005 -0.120* 0.004

Hispanic 0.009* 0.004 -0.013 0.007 0.015* 0.006 -0.015 0.008 0.036* 0.007

Multiracial -0.025* 0.005 -0.004 0.009 -0.044* 0.007 -0.027* 0.010 -0.015 0.009

American Indian -0.083* 0.008 -0.067* 0.017 -0.071* 0.012 -0.096* 0.018 -0.091* 0.013


Page 17 of 19


Variable

High School

All Subjects

High School

English I

High School

Mathematics

High School

Science

High School

Social Studies

Value

Standard

Error Value

Standard

Error Value

Standard

Error Value

Standard

Error Value

Standard

Error

Asian 0.057* 0.006 -0.022* 0.009 0.131* 0.009 0.069* 0.011 0.024* 0.010

Male 0.063* 0.002 -0.123* 0.003 0.003 0.003 0.104* 0.004 0.200* 0.003

LEP services recipient -0.077* 0.006 -0.154* 0.009 -0.006 0.009 -0.055* 0.013 -0.099* 0.012

Previous LEP services recipient 0.023* 0.007 -0.008 0.011 0.013 0.011 0.039* 0.014 0.022 0.012

Algebra II -0.389* 0.012 --- --- -0.405* 0.012 --- --- --- ---

English I 0.008 0.007 --- --- --- --- --- --- --- ---

Geometry -0.286* 0.009 --- --- -0.296* 0.010 --- --- --- ---

Biology -0.010 0.009 --- --- --- --- 0.926* 0.041 --- ---

Chemistry -0.572* 0.020 --- --- --- --- 0.392* 0.042 --- ---

Physical Science 0.304* 0.014 --- --- --- --- 1.243* 0.040 --- ---

Physics -0.960* 0.038 --- --- --- --- --- --- --- ---

Civics/Economics -0.014 0.008 --- --- --- --- --- --- 0.098* 0.008

U.S. History -0.103* 0.010 --- --- --- --- --- --- --- ---


Students per classroom -0.002* 0.000 0.001 0.000 -0.002* 0.001 -0.004* 0.001 -0.003* 0.000

Classroom ability dispersion 0.036* 0.010 0.022 0.014 0.047* 0.017 0.060* 0.021 0.060* 0.016

Advanced curriculum 0.127* 0.006 0.085* 0.008 0.235* 0.012 0.117* 0.015 0.096* 0.009

Remedial curriculum 0.011 0.017 0.005 0.010 0.014 0.037 -0.068 0.067 0.047 0.044


Page 18 of 19


Variable

High School

All Subjects

High School

English I

High School

Mathematics

High School

Science

High School

Social Studies

Value

Standard

Error Value

Standard

Error Value

Standard

Error Value Standard

Error Value

Standard

Error


School size (per 100) 0.010* 0.002 0.003 0.002 0.008* 0.004 0.002 0.004 0.017* 0.004

School size squared -0.000* 0.000 -0.000 0.000 -0.000 0.000 0.000 0.000 -0.000* 0.000

Total per-pupil expenditures ($100s) 0.000 0.000 0.001* 0.000 -0.000 0.000 -0.000 0.000 0.000 0.000

Average teacher supplement ($100s) 0.001* 0.000 0.001* 0.000 0.000 0.001 0.001 0.001 0.003* 0.001

Short-term suspension rate (per 100

students) -0.000* 0.000 -0.000 0.000 -0.000 0.000 -0.001* 0.000 -0.001* 0.000

Violent acts rate (per 1000 students) -0.001* 0.000 -0.000 0.000 -0.000 0.001 -0.002* 0.001 -0.001 0.000

Free and reduced price lunch mean -0.000 0.000 -0.001* 0.000 0.000 0.000 0.000 0.001 -0.001 0.000

Black mean 0.001* 0.000 0.001* 0.000 0.000 0.000 0.001* 0.000 0.001* 0.000

Hispanic mean 0.003* 0.001 0.002* 0.001 0.003* 0.001 0.002 0.001 0.004* 0.001

Multiracial mean -0.002 0.003 0.006* 0.002 -0.003 0.003 0.000 0.005 -0.004 0.004

American Indian mean 0.001 0.001 -0.000 0.001 0.000 0.001 0.001 0.001 0.003* 0.001

Asian mean 0.003 0.002 0.002 0.001 0.005* 0.002 0.004 0.003 -0.000 0.002

Note: Teachers with less than 5 years of experience and teaching EOC tested course during the 2005-06, through 2009-10 school year.

*Indicates a given coefficient is significant at the .05 level.


Page 19 of 19

Necessary Information for Days Equivalency Calculations

Table 4: Calculating Days Equivalency

End of Grade Test Standard Deviations Average Yearly Gains

Elementary School Mathematics 8.554 6.143

Elementary School Reading 8.065 5.061

Middle School Mathematics 8.293 2.791

Middle School Reading 7.763 2.719

Days Equivalency Equation = (((Result value/100) x Standard Deviation) / (Avg. Yearly Gains))) x 180

Example for Elementary School Mathematics

Step One

Result value from institution comparison graph (ES and MS = 6.75) Standard Deviation (8.554) and Average Yearly Gains (6.143) from the table above

Step Two

Insert the result value into the days equivalency equation

(((6.75/100) x 8.554) / (6.143))) x 180 = 16.92 days of learning

Days Equivalency for High School and Middle Grades Science/Algebra

For elementary and middle grades mathematics and reading tests, Carolina Institute for Public

Policy (CIPP) can calculate days equivalency values because the tests are interval scaled and

students have prior test scores for the subject. In high school subjects and middle grades science

and algebra, however, prior test scores do not exist. Therefore, CIPP did not provide days

equivalency values for these tested subjects.

Carolina Institute for Public Policy

TechProgramReportCover2011Technical Report 2011ProgramReport_Final_7-21-2011.pdfBACK COVER_2

7hfkqlfdo 5hsruw 81& 7hdfkhu 3uhsdudwlrq ......july 2011 by gary t. henry, unc-chapel hill charles...

Documents