Should We Focus on the Should We Focus on the NCLB “Toolkit” or on NCLB “Toolkit” or on Improving Learning?Improving Learning?

Marshall S. Smith

November 30, 2006

American Enterprise Institute

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 2

• Excellent, thoughtful set of papers

• Conclusions – almost universal!!!!– Implementation weak– Frequency of use – small to miniscule– No discernable achievement

effect

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 3

What Did We Expect?What Did We Expect?

• Complicated, patchwork, non-systemic interventions

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 4

Moreover Moreover —— No Support from Prior Research No Support from Prior Research

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 5

Time to Stop Messing AroundTime to Stop Messing Aroundon the Marginson the Margins

• What do we know about achievement over past decades?

• What do we know about how to improve learning?

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 6

Recent Achievement TrendsRecent Achievement TrendsLook at NAEP subgroups for accurate

picture. Approximately 1994-2005. Both NAEPs.

Math gains large – 1 to 2+ grade levels.

Reading modest but significant. 0.7-1+ gl.

4th grade gains considerably larger than 8th grade.

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 7

Reading Main NAEP

Reading Longitudinal

Math Main NAEP

Math Longitudinal

2005-1994 gains

2004-1994 gains

2005-1992 gains

2004-1994 gains

National 5 818

10

White 5 9 19 10African

American 15 15 27 12

Hispanic American 15 19 24 20

NAEP Gains for Grade 4 (Main) and Age 9 (Longitudinal) – Reading and Mathematics (10-11 points = 1 grade

level)

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 8

Achievement Before and After Achievement Before and After Standards-Based ReformsStandards-Based Reforms (SBR)(SBR)

Compare time periods 1978-1994 with 1994-2005 — Longitudinal NAEP

Math — Gain slopes before and after SBR similar — slight edge to after SBR

Reading — Gain slopes considerably larger after SBR

SBR based on these data SBR is a qualified success

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 9

Achievement Before and After Achievement Before and After NCLBNCLB

• Compare achievement slopes 1994-2002 with 2002-2005

• Math slopes approximately the same

• Reading slopes greater before NCLB

• Jury out (need 2007 data) but certainly no evidence of success!

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 10

Main NAEP Main NAEP Reading Scale Scores – Grade 4Reading Scale Scores – Grade 4

GroupsYears Gains

1994 2002 20052002-1994

2005-2002

Nation 214 219 219 +5 0

White 224 229 229 +5 0

African American 185 199 200 +14 +1

Hispanic American 188 201 203 +13 +2

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 11

Main NAEP Main NAEP Reading Scale Scores – Grade 8Reading Scale Scores – Grade 8

Years Gains

Groups 1994 2002 20052002-1994

2005-2002

Nation 260 264 262 +4 -2

White 267 272 271 +5 -1

African American 236 245 243 +9 -2

Hispanic American 243 247 246 +4 -1

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 12

Summing Up So FarSumming Up So Far

• Math and reading gains larger than many believe. Substantial gains in 4th grade in reading and math during SBR years, less but still important gains in 8th grade.

• Gaps closing slowly over past 12 years in 15 of 16 comparisons in NAEP. Between 10% and 30% from 1994-2005. Not fast enough.

• SBR a qualified success. Three of four cases gain slopes are greater after the SBR reforms started than before.

• No evidence of NCLB effectiveness. Substantial decline in slope of gains since NCLB passed for reading in both grades – no difference for math.

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 13

One OptionOne OptionStay the Course – with minimal correctionStay the Course – with minimal correction

• Standards-based reform works.

• It is too early to see results of NCLB (“toolkit”, tough accountability and Reading First). We need more time but we know they work.

• Therefore up the ante – tell locals to implement better -- make sanctions tougher on “misuse” of NCLB “tools” -- increase intensity of accountability to make sure results come faster.

• Create and require national tests because, I guess, they believe, they will make everyone work harder and learn more.

Some folks argue:

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 14

What Should We Do?What Should We Do?• Don’t stay the course! Take bold steps but

build on strength.

• Most educational policy is on the margin – little programs, little add-ons, create chaos and have no effects. Throw them out!

• System wants us to continue playing on the margins – this satisfies adults, special interests, and other inside beltway inhabitants.

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 15

Think About Improving the Think About Improving the System to Improve LearningSystem to Improve Learning

• Learning = f(S, C, M, T)– System– Content– Motivation– Time

• Shoot for big effects

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 16

SystemSystem

• Use positive incentives throughout the system – reward rather than punish – encourage creativity and initiative rather than regulate rigidity.

• Support information-based continuous improvement at all levels.

• Retain SBR teacher qualification requirements, dis-aggregation, school, district and state reports, increase transparency.

• Support longitudinal student and teacher-based data systems.

• Hold accountable those who make decisions – not only those who carry them out.

• Remove add-ons!

Remember – the point is to help schools improve and children learn!

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 17

ContentContent• Focus on Language, Language, Language!!!

• Start early and continue throughout – preschool and beyond with good role models, lots of oral language, comprehension, complex sentence structure, lots and lots of reading, academic language.

• E.D. Hirsch, Catherine Snow,etc.

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 18

Motivation and Desire to LearnMotivation and Desire to Learn• Ensure healthy children –

teeth, eyes, ears, childhood disease.

• Increase Music, Art, Learning by Doing, enhance creativity and learning how to learn. Analysis, synthesis. Give students more control over their learning.

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 19

TIMETIME• Create incentives to extend

time of the regular school day and year (learn from KIPP and others)

• Accelerate learning for those who are bored — using technology creatively

Marshall SmithThe William and Flora Hewlett FoundationNovember 30, 2006 20

CLIMB Out of the BOXCLIMB Out of the BOX

Don’t settle for small effects!