Unintended Consequences of an Algebra-for-all Policy on High-skill

Students:The Effects on Instructional Organization and

Students’ Academic Outcomes

Takako Nomi

Consortium on Chicago School Research at the University of Chicago

Funded by the Institute for Education Sciences

March 2010

2

Movement for More Rigorous Course Requirements

National Context• Recognition that high schools are not sufficiently preparing students for

college or workforce• National movements calling for increasing graduation requirements

College-prep curriculum is in place in 20 states, and more states and districts are raising graduation requirements in near future

Chicago Context• Chicago increased graduation requirements in 1997

Requiring college-prep curriculum for all students Ending remedial coursework All students required to take Algebra in 9th grade, followed by geometry and

algebra II

3

Prior ResearchResearch suggests that rigorous coursework improves students’

academic outcomes• Constrained academic curriculum (private vs. public schools)• Tracking• High school coursework (e.g., AP courses) tied to college outcomes

Limitations in prior research• Not examining the effects of a policy mandating college-prep curriculum• Selection bias among schools & among students within schools • Not examining unintended consequences of such a policy

Our recent work addressed some of these limitations (Allensworth, et al. 2010)

4

Policy effects in Chicago

The effects of mandating a college-prep curriculum for the policy target students--average and lower ability students who would take remedial math in the absence of the policy

When schools increased algebra enrollment: More students took algebra and earned credits More students failed in math

• Students were more likely fail in Algebra than in remedial Math Test scores unchanged

Limitations: Unintended consequences on non-target students

5

Unintended ConsequencesNon-target students• High-skill students who would take Algebra regardless of the policy• The policy would not affect their course enrollment, but might affect their

academic outcomes

Instructional Organization: How classrooms are organized for instructional purposes

• Student diversities in academic backgrounds• How to organize instruction to meet the need of all students with varying skills• Tracking/ability grouping—Tailored instruction

Current policy discussions do not address how universalizing a college-prep curriculum may affect instructional organization

• Classroom academic composition affects students’ outcomes (e.g., content difficulty and instructional pace)

6

Research Questions1) To what extent did a policy that required algebra for all students in

ninth grade affect classroom academic composition?• Schools might maintain the existing practice

– Classroom academic composition remain the same

• Schools might reorganize classrooms, incorporating low-skill students who would otherwise take remedial math into Algebra classes– Classrooms become more mixed-ability– Peer ability levels would decline for high-skill students

2) For high-skill students who were not targeted by the policy, how did the policy affect their academic outcomes?• If peer ability levels declined, were their outcomes negatively affected by

the policy?

7

DataThe population of first-time 9th grade cohorts in Chicago public high

schools from 1994-95 to 1999-2000 (6 cohorts) Schools in existence pre- and post-policy Total N = 104,000 in 57 schools 20,215 high-skill students

CCSR Archive Semester course transcripts Complete administrative records with demographic information Elementary and high school achievement test scores

8

Variables

CovariatesStudent Level

8th grade math ability Demographics (Gender, Age, poverty,

social status, Race/ethnicity, Mobility prior to high school)

Special education status

School Level Academic composition (average

ability levels) in the base year

Cohort Level Year-to-year changes in

academic composition

9th-grade Math test scores

Student Outcomes

9

DesignsTwo comparisons Short interrupted time series (Abrupt changes in Algebra enrollment)

Changes in outcomes between pre-policy and post-policy cohorts Problem with interrupted time series—other changes unrelated to the policy may

affect outcome changes

Within-cohort comparisons Comparing outcome changes between schools affected by the policy and schools

unaffected by the policy Only schools that offered remedial math pre-policy were affected by this policy,

while all schools were affected by other changes Some schools offered algebra for all students pre-policy, including very low ability st

udents (control schools)

10

AssumptionsNo systematic differences between schools affected and

unaffected by the policy in a way that are related to outcome trends

• Tested in a number of ways… After controlling for students’ incoming abilities, measured school

characteristics did not explain why some schools had higher pre-policy algebra enrollment

Trends in students’ academic composition were similar between schools affected and unaffected by the policy

Pre-policy outcome trends were similar between schools affected and unaffected by the policy though their outcomes differed in the base year

11

Statistical Models Policy effect on students outcomes

Level 1 (Students)Yijk= π0jk + π1jk(math ability)ijk + π2jk(math ability2)ijk + Ʃπik(X)ijk + eijk

Level 2 (Cohorts)π0jk = β0k + Ʃβk(cohort year)jk +β6k(cohort ability)jk + rjk Level 3 (Schools)For intercept (pre-policy average outcomes) ,β0k = γ00 + γ01(Affected)k + u0k

For each cohort-year slope (Β1k through Β5k ),Β1k = γ10 + γ11(Affected)k + u1k

12

Overall Changes in Classroom Academic Composition

Unaffected Schools (full Algebra enrollment pre-policy)

*Controlling for cohort average abilities

13

Overall Changes in Classroom Academic Composition

Unaffected Schools (full Algebra enrollment pre-policy)

Affected Schools (increased Algebra enrollment post-policy)

*Controlling for cohort average abilities

14

Policy Effects on Classroom Academic Composition among high ability students

Est. Intercept (yr94) 0.04

Affected 0.18***

Yr95(pre) 0.01Affected -0.04

Yr96(pre) 0.03Affected -0.02

*Controlling for cohort average abilities and student characteristics

15

Policy Effects on Classroom Academic Composition among high ability students

Est. Intercept (yr94) 0.04

Affected 0.18***

Yr95(pre) 0.01Affected -0.04

Yr96(pre) 0.03Affected -0.02

*Controlling for cohort average abilities and student characteristics

16

Policy Effects on Classroom Academic Composition among high ability students

Est. Intercept (yr94) 0.04

Affected 0.18***

Yr95(pre) 0.01Affected -0.04

Yr96(pre) 0.03Affected -0.02

yr97(post) -0.01Affected -0.04

yr98(post) 0.11Affected -0.17**

yr99(post) 0.07Affected -0.14*

*Controlling for cohort average abilities and student characteristics

17

Policy Effects on Test scores for High-ability Students

Model 1

Intercept (yr94) 53.15*** Affected 1.80*

Yr95(pre) -0.25 Affected -0.05

Yr96(pre) 2.01* Affected -0.15

18

Policy Effects on Test scores for High-ability Students

Model 1

Intercept (yr94) 53.15*** Affected 1.80*

Yr95(pre) -0.25 Affected -0.05

Yr96(pre) 2.01* Affected -0.15

yr97(post) 1.00 Affected -0.40

yr98(post) 4.74*** Affected -1.85~

yr99(post) 5.29*** Affected -2.53*

19

Policy Effects on Test scores for High-ability Students

Model 1 Model 2 (class composition)

Intercept (yr94) 53.15*** 52.95 Affected 1.80* 0.32

Yr95(pre) -0.25 -0.06 Affected -0.05 -0.67

Yr96(pre) 2.01* 1.09 Affected -0.15 1.20

yr97(post) 1.00 0.66 Affected -0.40 0.71

yr98(post) 4.74*** 3.86*** Affected -1.85~ -0.76

yr99(post) 5.29*** 4.53*** Affected -2.53* -1.00

20

Trends in math scores

Policy

21

SummaryCurricular structure shapes how classrooms are organized

Schools that offered remedial math pre-policy used greater levels of sorting & more homogeneous grouping (tracking)

• High-skill students attend classes with other high-skill students, and vise versa for low-skill students

• In schools that did not offer remedial math, students attend more mixed-ability classrooms

Algebra-for-all policy eliminated curriculum differentiation, but it also created more mixed ability classrooms For high-skill students, even though the policy did not affect their course

enrollment, it negatively affected their academic outcomes due to declines in peer ability levels

• Instruction targeted at the middle students in the class

22

Implication/future researchPolicy Implication Curricular policies need to consider multiple ways that affect students

• Simply raising graduation requirements is not enough

How can we raise standards/expand opportunities without comprising excellence ?

• Chicago implemented double-period Algebra in ‘03 (Nomi & Allensworth, 2009)

• This intensified tracking, but provided supports for struggling students

• This approach was successful in raising test scores for all students (but not for Algebra grades)

Future research The Algebra-for-all policy might have benefited a subgroup of students

• Low-ability students who would take remedial math vs. those who would take algebra in the absence of the policy

23

Algebra enrollment increased immediately with the policy

High-skill students took algebra regardless of the policy

The policy did not affect their course enrollment

Changes occurred immediately following policy

199419951996

1997 19981999

200020012002

20032004

Pre-Policy Early Post-Policy Mid Post-Policy Late Post-Policy

COHORTS

Pre-Policy Algebra Enrollment Not Determined by School Composition

Schools’ pre-policy algebra enrollment by school mean math ability for low-ability students

Among both schools with high- or low-average abilities, some had almost 100% algebra enrollment pre-policy.

These schools were not affected by the policy.

High degree of variation in algebra enrollment pre-policy among schools with the same average ability

Perc

ent o

f low

-abi

lity

stud

ents

enr

olle

d in

A

lgeb

ra

School average math ability

Each dot (+) represents a

school