External Curriculum ReviewBackground

In 2010, the Maryland State Board of Education adopted the Common Core State Standards,

later known as the Maryland College and Career Ready Standards. As a result, MCPS began

revising the written, taught, and learned curriculum to align with the Maryland College and

Career Ready Standards. MCPS developed the curriculum to respond to the existing conditions,

feedback, and interests prevalent that time, which included: using an internally developed online

platform to house curriculum and resources; leveraging teachers currently in, or recently out of,

the classroom as writers of curriculum, materials, and assessments; and responding to strong

interest from stakeholders in using lesson seeds and sample learning tasks as foundational

elements to the curriculum.

The revised curriculum, referred to as Curriculum 2.0, has been implemented over a multiyear

period. MCPS is in the fifth year of full implementation of Grades Pre-K–5 in English

Language Arts (ELA) and Mathematics, and the third year of full implementation in

Grades 6 through 8. Board of Education Policy IKA, Curriculum, encompasses the

written, taught, and learned curriculum. The policy requires reviews of curriculum

content areas on five-year cycles. The elementary curriculum met the five-year review

point this year, creating an opportune time for an external review of the curriculum.

The Research Team

In Spring 2017, Montgomery County Public Schools (MCPS) awarded a contract to the Johns

Hopkins School of Education to evaluate Curriculum 2.0. The Johns Hopkins Institute for Education

Policy (“the Institute”) served as project lead. The Institute brings deep experience in the curriculum

domain to this work; its senior partners are currently advising seven states and numerous districts on

curriculum policy. The Johns Hopkins Center for Research and Reform in Education (“CRRE”),

which has directed multiple I3 grants from the USDOE, served as statistical specialists in both the

project design and the survey analysis. The Institute brought additional experts to complement its

team, specifically Student Achievement Partners (the organization that lead-authored the Common

Core State Standards); Lengel Educational Consulting, which has advised more than 70 school

districts on issues of pedagogy and instruction; and StandardsWork, which has created professional

learning tools and materials for school districts, state agencies, and national organizations for

25 years. In the remainder of this report, reference to “the Institute team” represents the collective

judgment and expertise of all partners on this project.

Rationale for the Audit

Curriculum matters. High-quality research suggests that using best-in-class instructional materials

can improve student learning even more than other, more well-known, interventions such as

expanding preschool programs, giving merit pay to successful teachers, decreasing class sizes, or

increasing the number of charter schools in a district.1 Despite this, few states and districts view the

curriculum as an important policy lever for change.

1 (Chingos and Whitehurst 2012), (Boser, Chingos, and Straus 2015)

MCPS is an exception. MCPS has a longstanding history of designing and implementing instructional

materials that challenge students, empower educators, and narrow achievement gaps. Curriculum 2.0

was one of the earliest curricula developed in the Common Core era. The curricular landscape has

changed since then; research on the effects of standards is significant; new instructional models have

entered the market.

In Montgomery County, curriculum is routinely reviewed in alignment with Board of Education

policy (every five years), as are external resources that are available on the market. Coinciding with

the arrival of a new superintendent and after approximately five years of implementation of

Curriculum 2.0, an external review of the written, taught, and learned curriculum was conducted.

This periodic review provides for internal controls that cause MCPS to reflect upon and modify

curriculum as conditions demand. MCPS is not alone in the effort to ensure that curriculum is

regularly modified and aligned with state or national standards. Many large districts struggle with

the decision to continually develop and update curriculum or purchase curriculum, which is a

common occurrence across the United States. The current project is designed to review MCPS’s

written, taught, and learned curriculum in light of the state’s academic standards, the experience of

other comparable districts, and the district’s expectations for excellence and equity.

Attachments (click below)1. Memo Curriculum Review (PDF)2. Executive Summary MCPS (PDF)

3. Curriculum Review PowerPoint (PDF)

DISCUSSION

Office of the Superintendent of Schools

MONTGOMERY COUNTY PUBLIC SCHOOLS

Rockville, Maryland

March 22, 2018

MEMORANDUM

To: Members of the Board of Education

From: Jack R. Smith, Superintendent of Schools

Subject: Curriculum Review

Background

Montgomery County Public Schools (MCPS) is committed to improving teaching and learning

and ensuring that all students achieve at the highest levels, prepared to succeed in postsecondary

study, career, and community. Our student achievement data identify significant gaps

in performance for our Black or African American students, Hispanic/Latino students, students

in poverty, as well as students receiving special education and English for Speakers of Other

Languages services. The pursuit of closing these gaps and improving educational outcomes

for all students fuels MCPS’ dedication to continuous improvement. Research suggests that using

best-in-class instructional materials may improve student learning even more than other more

well-known interventions. This reinforces the sense of urgency to ensure that instructional

materials and curriculum are of the highest quality and aligned to changing standards.

During the 2009–2010 school year, MCPS initiated a process to revise the existing curriculum

through a stakeholder feedback process. New curriculum development for elementary school

began during the 20019–2010 school year. In June 2010, the Maryland State Board of Education

adopted the Common Core Standards, later known as the College and Career Ready Standards.

As a result, MCPS shifted away from the initial curriculum development process and began

revising the written, taught, and learned curriculum to align with the new standards. MCPS

developed the curriculum to respond to the existing conditions, feedback, and interests prevalent

at that time, which included: using an internally developed online platform to house curriculum

and resources; leveraging teachers currently or recently out of the classroom as writers

of curriculum, resources, and assessments; and responding to strong interest from stakeholders

in using lesson seeds and sample learning tasks as foundational elements to the curriculum

as opposed to highly prescriptive curricula and daily lessons.

Members of the Board of Education 2 March 22, 2018

The revised curriculum, referred to as Curriculum 2.0, has been implemented over a multiyear

period. MCPS is in the fifth year of full implementation of Grades Pre-K–5 in English Language

Arts (ELA) and Mathematics, and the third year of full implementation in Grades 6–8.

Staff members have shared the strengths and challenges of the curriculum, the complications

of receiving the written curriculum through an online platform, as well as the varying amount

of professional development that has been provided during often difficult budgetary times.

The curriculum is predicated upon the expectation that the taught curriculum is delivered

in a manner that stresses critical and creative thinking as well as deep understanding. Board Policy

IFA, Curriculum, explicitly articulates the link between the written, taught, and learned

curriculum. In addition, the policy requires reviews of all curriculum content areas on five-year

cycles. This year marks the five-year review period for the elementary curriculum, thus making

it the perfect time to reflect upon lessons learned and how to best position ourselves for the next

iteration of curriculum and instructional resources in the coming years.

To better understand how we may build on what is working and identify areas for improvement,

MCPS contracted with Johns Hopkins University (JHU) to conduct a comprehensive review

and analysis of the MCPS written, taught, and learned curriculum. The review is intended

to accomplish the following objectives: (a) assess the alignment of the MCPS written, taught,

and learned curriculum with the Maryland College and Career Ready Standards (which incorporate

the Common Core State Standards); and (b) provide technical advice and expertise to identify

possible actions to address areas in need of improvement. The comprehensive review encompasses

an analysis of three aspects of the MCPS curriculum—written, taught, and learned.

Written Curriculum Review

The written curriculum review is designed to determine whether instructional materials are aligned

to the Maryland College and Career Ready Standards. The process involved a comprehensive

review of the overall structure of Curriculum 2.0, including, for example, the scope and sequence

as well as a review of a subset of the units in one grade per grade band (pre-K–2, 3–5, and 6–8)

in both ELA/Literacy and mathematics up to Algebra 1. The reviewers assessed the availability,

alignment, and quality of embedded supports within the curriculum for second language learners

and other special populations.

Taught Curriculum Review

Teacher Survey

A teacher survey was conducted to gather information on teacher/user experience with Curriculum

2.0, including its accessibility on the MCPS platform. The survey focused on instructional

materials and the taught curriculum, and any discrepancies between adopted instructional materials

and instruction in the classroom.

Members of the Board of Education 3 March 22, 2018

Observations and Focus Groups

To identify the inconsistencies between the written and taught curriculum, the JHU team

conducted focus groups with a variety of stakeholders, including a representative sampling

of classroom teachers and a range of students. Approximately 80 classrooms in 20 elementary

and middle schools were observed.

Learned Curriculum Review

Student artifacts gathered during classroom observations were analyzed to review the alignment

of the written curriculum with classroom instruction. The JHU team also analyzed

district-developed and external assessment student results to identify areas of strength and areas

for improvement.

Timeline

The review began in July 2017 and continued through January 2018. The JHU team spent

the last six weeks completing their report. MCPS staff members reviewed the findings

and recommendations and have developed a preliminary action plan in response to the report.

Despite the timing of the release of the report, while we are deep into the planning process for next

school year, it is imperative we take action to address the report recommendations with the start

of the 2018–2019 school year. Anticipating the need to address any findings, the Board included

funding to respond to the curriculum review in the Fiscal Year 2019 budget. With five months

before the start of the new school year, and funding in the budget to support curriculum changes,

we are well positioned to initiate a change process this summer and into next year, in keeping with

our shared sense of urgency to provide students with the highest quality educational experience.

Next Steps

MCPS has had a long tradition of writing curriculum, creating and delivering lesson resources,

and developing assessments, through the expertise of teachers and central services staff.

Overall findings and recommendations of this review reinforce the notion that in order for MCPS

to maintain the highest quality instructional materials for teachers and students, the time is right

for MCPS to move away from a model that relies on utilizing central services staff and teachers

for writing curriculum and assessments, to a model based on adopting external curriculum

developed by curriculum and assessment experts. Externally developed curricula also provide the

benefit of frequent and ongoing modifications and online platforms that are regularly updated

based on trends in technology and user feedback.

Through external evidence-based research and reviews, several ELA and mathematics externally

developed curricula are highly rated for their alignment with the Common Core State Standards.

This evidences a significant development since the early days of the Common Core when there

were few external products of sufficiently high quality. Well-aligned curricular products were

Members of the Board of Education 4 March 22, 2018

not available when Curriculum 2.0 development was initiated in the 2009–2010 school year.

The more recent externally developed curricula offer strong advantages in terms of readily

available materials for struggling or advanced students, as well as various resources for special

needs and English Language Learners.

Today, we will present the Board with an update on this changing philosophy, moving from

an internally developed curriculum to an externally adopted curriculum that already is highly

rated and proven. We also will hear from JHU staff members regarding their findings

and recommendations as well as their perspective on the next steps for MCPS.

In addition, the MCPS team will share a proposed timeline for how to begin to transition away

from Curriculum 2.0 toward an externally purchased curriculum that brings its own platform,

constantly provides updated materials and resources written by experts, and employs an approach

that aligns with our core values and instructional priorities. As we transition, feedback from

stakeholders will be another important consideration as new materials are selected.

As recommended in the review, MCPS will transition over a multiyear period to ensure effective

implementation and support to schools. The review of the written, taught, and learned curriculum

offers MCPS a tremendous opportunity to learn and grow, and most importantly, positions

us to improve teaching and learning for ALL our students.

JRS:MVN:EJL:mec

March 2018 Montgomery County Public Schools: Curriculum Review and Analysis Summary and Recommendations Submitted To: Montgomery County Board of Education

Review Participants: The Johns Hopkins Institute for Education Policy (Project lead) The Johns Hopkins Center for Research and Reform in Education Student Achievement Partners Lengel Educational Consulting StandardsWork

MCPS: Curriculum 2.0 Evaluation Summary and Recommendations 2

Acknowledgments We would like to express our sincere appreciation to the leadership of the Montgomery County Public

Schools (MCPS), in particular Dr. Jack Smith, Superintendent; Dr. Maria Navarro, Chief Academic

Officer; Sharron Steele, Executive Director of the Chief Academic Officer’s office; and the entire

central office leadership at Montgomery County Public Schools (MCPS), for enabling a full and

rigorous review of the MCPS 2.0 Curriculum.

We would also like to thank the staff, teachers, and principals whose classes and schools we visited,

who participated in the focus groups, and/or who responded to the online surveys.

The Johns Hopkins Institute for Education Policy (The Institute) assembled a team of participants to

engage with this study. In addition to the Institute’s own experts - Dr. David Steiner, Dr. Ashley

Berner, and Dr. Alanna Borklund-Young – the team included Dr. Steven Ross, Dr. Jennifer Morrison,

and Dr. Betsy Wolf of the Center for Research and Reform in Education; Ms. Barbara Davidson,

Executive Director of StandardsWork; Ms. Susan Pimentel and Mr. Jason Zimba, Founding Partners,

and Dr. David Liben, Dr. Shelbi Cole, Ms. Marni Greenstein, Ms. Katie Keown, and Ms. Beth

Cocuzza, Content Specialists, of Student Achievement Partners; and Mr. Jim Lengel and Ms. Kathi

Lengel, partners, Lengel Educational Consulting. Special thanks go to Ms. Michele Mengel, the

Institute’s Administrative Coordinator, and to Ms. Susanna Elliott, the School of Education’s Senior

Grants and Contracts Analyst.

MCPS: Curriculum 2.0 Evaluation Summary and Recommendations 3

Executive Summary Background In 2010, the Maryland State Board of Education adopted the Common Core State Standards,

later known as the Maryland College and Career Ready Standards. As a result, MCPS began

revising the written, taught, and learned curriculum to align with the Maryland College and

Career Ready Standards. MCPS developed the curriculum to respond to the existing conditions,

feedback, and interests prevalent that time, which included: using an internally developed online

platform to house curriculum and resources; leveraging teachers currently in, or recently out of,

the classroom as writers of curriculum, materials, and assessments; and responding to strong

interest from stakeholders in using lesson seeds and sample learning tasks as foundational

elements to the curriculum.

The revised curriculum, referred to as Curriculum 2.0, has been implemented over a multiyear

period. MCPS is in the fifth year of full implementation of Grades Pre-K–5 in English

Language Arts (ELA) and Mathematics, and the third year of full implementation in

Grades 6 through 8. Board of Education Policy IKA, Curriculum, encompasses the

written, taught, and learned curriculum. The policy requires reviews of curriculum

content areas on five-year cycles. The elementary curriculum met the five-year review

point this year, creating an opportune time for an external review of the curriculum.

The Research Team In Spring 2017, Montgomery County Public Schools (MCPS) awarded a contract to the Johns

Hopkins School of Education to evaluate Curriculum 2.0. The Johns Hopkins Institute for Education

Policy (“the Institute”) served as project lead. The Institute brings deep experience in the curriculum

domain to this work; its senior partners are currently advising seven states and numerous districts on

curriculum policy. The Johns Hopkins Center for Research and Reform in Education (“CRRE”),

which has directed multiple I3 grants from the USDOE, served as statistical specialists in both the

project design and the survey analysis. The Institute brought additional experts to complement its

team, specifically Student Achievement Partners (the organization that lead-authored the Common

Core State Standards); Lengel Educational Consulting, which has advised more than 70 school

districts on issues of pedagogy and instruction; and StandardsWork, which has created professional

learning tools and materials for school districts, state agencies, and national organizations for

25 years. In the remainder of this report, reference to “the Institute team” represents the collective

judgment and expertise of all partners on this project.

Rationale for the Audit Curriculum matters. High-quality research suggests that using best-in-class instructional materials

can improve student learning even more than other, more well-known, interventions such as

expanding preschool programs, giving merit pay to successful teachers, decreasing class sizes, or

increasing the number of charter schools in a district.1 Despite this, few states and districts view the

curriculum as an important policy lever for change.

1 (Chingos and Whitehurst 2012), (Boser, Chingos, and Straus 2015)

MCPS: Curriculum 2.0 Evaluation Summary and Recommendations 4

MCPS is an exception. MCPS has a longstanding history of designing and implementing instructional

materials that challenge students, empower educators, and narrow achievement gaps. Curriculum 2.0

was one of the earliest curricula developed in the Common Core era. The curricular landscape has

changed since then; research on the effects of standards is significant; new instructional models have

entered the market.

In Montgomery County, curriculum is routinely reviewed in alignment with Board of Education

policy (every five years), as are external resources that are available on the market. Coinciding with

the arrival of a new superintendent and after approximately five years of implementation of

Curriculum 2.0, an external review of the written, taught, and learned curriculum was conducted.

This periodic review provides for internal controls that cause MCPS to reflect upon and modify

curriculum as conditions demand. MCPS is not alone in the effort to ensure that curriculum is

regularly modified and aligned with state or national standards. Many large districts struggle with

the decision to continually develop and update curriculum or purchase curriculum, which is a common

occurrence across the United States. The current project (RFP 4395.1) is designed to review MCPS’s

written, taught, and learned curriculum in light of the state’s academic standards, the experience of

other comparable districts, and the district’s expectations for excellence and equity.

Summary of the Curriculum 2.0 Audit Process The Institute team undertook a 360° evaluation of Curriculum 2.0, involving multiple, overlapping

perspectives and data collections. These can be summarized as follows:

1. A formal alignment evaluation comparing Curriculum 2.0 materials in math and ELA,

Kindergarten through 8th-grade, against the multiple requirements of the Maryland state

standards.

2. An evaluation of classroom practices, involving on-site classroom observations; analyses of

student work; survey data of the views of stakeholders; and focus groups with teachers,

principals, and central office staff. The purpose of this review was to listen to those who

worked with Curriculum 2.0 in schools – and to examine how the use of Curriculum 2.0 in

MCPS classrooms influenced the delivery of standards-based education.

3. An evaluation of MCPS 3rd- through 8th-grade PARCC scores over the past three school years.

The purpose was to identify the specific grades, subjects, and student groups where

proficiency levels were outliers in comparison to MCPS students in surrounding grade levels.

The evaluation also compared MCPS PARCC scores to comparison school districts with

identifiable sub-groups of similar proportion and demography to those in MCPS.2

2 In Maryland: Prince George’s County, Baltimore County; Anne Arundel County, and Howard County. In Colorado:

Jefferson County and Cherry Creek County. Additional: Washington, D.C.

MCPS: Curriculum 2.0 Evaluation Summary and Recommendations 5

Overall Conclusions

1. The evaluation surfaced several positive aspects of Curriculum 2.0. Among them:

Most of the observed lessons, in both ELA and math, drew upon the learning goals from

Curriculum 2.0.

Math classrooms use resources from Curriculum 2.0 the majority of the time.

The middle school math materials of Curriculum 2.0 approach alignment to the high-level

requirements of the Maryland College and Career Ready Standards.

2. The evaluation also surfaced a number of consistent challenges associated with

Curriculum 2.0 that were confirmed across each domain of the inquiry. These challenges are

elaborated in the section below.

The Evaluation Findings Formal alignment analysis. The instrument of analysis was the Instructional Materials Evaluation

Tool (IMET), created by Student Achievement Partners, which is regarded as the national

gold-standard instrument for curricula alignment analysis. EdReports, for instance, draws upon the

IMET tool in all of its published curricular evaluations.

Based on the materials reviewed, the ELA curriculum in K through 8th grade must be better

aligned to central features of the Maryland College and Career Ready Standards.

The math curriculum in K through 5th grade must be better aligned to central features of

the Maryland College and Career Ready Standards.

Teachers expressed concerns about Curriculum 2.0. They find the curriculum misaligned

to the needs of many of their students, difficult to navigate on the technology platform and

requiring more opportunities for students to develop a depth of understanding.

While most teachers use Curriculum 2.0, they supplement it with additional lessons and

assignments from other sources.

Student work samples. The instrument of analysis was an MCPS-appropriate version of the Student

Work Analysis Tool that is part of Student Achievement Partners’ Instructional Practice Toolkit

(IPT). In total, the team reviewed 36 different assignments and 530 student work samples in math,

and 34 assignments and 455 student work samples in ELA.

Overall, the student work samples indicate a misalignment between the learning standards and most

student assignments.

Math:

Student work samples did not consistently show mastery of the learning standard. Student

work samples show that fewer than a third of students master their assignments in either

ELA or math, although mastery in mathematics is higher than in ELA.

In K through 2nd-grade math, student work samples did not consistently show full or close

to full mastery of the targeted standard.

MCPS: Curriculum 2.0 Evaluation Summary and Recommendations 6

ELA:

ELA lessons did not consistently show alignment to the targeted standard. In the analysis

of student work samples, fewer than a quarter of students show complete mastery of the

assignments’ targeted standards.

In the majority of student work samples analyzed, students did not consistently show

mastery in the comprehension of their texts.

Focus groups analysis. The instrument of analysis was an MCPS-appropriate version of the Common

Core Knowledge and Practice Survey developed by Student Achievement Partners. In total, the

research team conducted 52 focus groups and interviews at 20 MCPS elementary and middle schools

with 324 educators – including both teachers and central staff - collecting 2,441 comments.

The focus groups reflect agreement about the curriculum’s strengths and weaknesses.

Teachers find the curriculum to contain too many required lessons and assessments.

Overall, the participating teachers do not have a positive view of Curriculum 2.0.

Teachers note several missing elements that are essential to understanding upcoming

lessons in both math and ELA. Some teachers attribute this problem to the pace of the

curriculum, which does not allow for practice. In fact, the need for students to practice

was mentioned repeatedly.

Insufficient knowledge-building in ELA. Many teachers feel the curriculum “miss[es] the

opportunity to build knowledge” across disciplines.

While most teachers use Curriculum 2.0 every day, they supplement it with additional

lessons and assignments from other sources.

Teachers expressed concern that Curriculum 2.0 does not adequately meet the needs of

their special education students or students with limited English proficiency.

Classroom observations. The instrument of analysis was the Instructional Practice Guide (IPG)

developed by Student Achievement Partners. In total, the research team conducted observations in

203 schools across the district, encompassing 14 elementary schools and 6 middle schools. The team

observed 77 classrooms for 20–30 minutes each: 38 ELA classrooms4 and 39 math classrooms.5

Overall, classroom observations show that teachers struggle to adapt lessons to students’ needs,

especially in ELA.

Math:

Many teachers had difficulty making mathematical content clear in their lessons and

making “the mathematics of the lesson explicit through the use of explanations,

representations, tasks, and/or examples”. Teachers across all elementary- and middle-

school grades make mathematical errors; significant errors occurred in 18 percent of the

classrooms, and minor mathematical errors in 23 percent of the classrooms.

3 As stated above, MCPS selected 8 of these schools based on their demographics and historical performance; the research

team selected the remaining 12 using principal components analysis and randomized stratified sampling methods. 4 The grade levels of the observed classrooms are as follows: 13 in K–2; 13 in 3–5; and 12 in 6–8. 5 The grade levels of the observed classrooms are as follows: 14 in K–2; 13 in 3–5; and 12 in 6–8.

MCPS: Curriculum 2.0 Evaluation Summary and Recommendations 7

ELA:

Text complexity. Students are not consistently exposed to grade-level texts across

3rd through 8th grade. Only approximately half of the texts observed are at or above

grade-level, and half are below grade-level.

Foundational skill instruction. Fewer than half the lessons in K through 2nd grade address

foundational skills. Five out of 11 lessons in K through 1st grade address foundational

skills, and none of the 2nd-grade lessons address foundational skills. The foundational

skills primarily target high-frequency words.

Textual evidence. Most questions and tasks are not text-dependent and do not require

evidence to complete.

Teacher survey. The survey questions were derived from the RAND Corporation’s American

Teacher Panel study with MCPS-appropriate modifications. In total, 1,852 teachers responded to the

survey. Fifty-six percent of respondents were from elementary schools, 44 percent from middle

schools. The respondents were seasoned educators, having taught an average of 15 years.

ELA teachers use non-Curriculum 2.0 materials more frequently than they do

Curriculum 2.0 materials. Ninety percent report using materials they personally

developed, and 70 percent used materials they personally selected from the Internet, daily

or almost daily.

While math teachers use Curriculum 2.0 materials more frequently than do ELA teachers,

approximately the same percentage of math teachers (87 percent) use self-developed

materials, and only slightly fewer use materials selected from the Internet, daily or almost

daily.

Teachers do not believe Curriculum 2.0 adequately meets the needs of special education

or ELL students.

Teachers report that they spend time on reading and skill development, but this was not

documented in the classroom observations.

MCPS’s PARCC results

Proficiency levels in 3rd- and 6th-grade ELA are lower than those in surrounding grades.6

“Control districts do not uniformly show this performance ‘dip.’

Proficiency levels among students with limited English are lower in each grade level than

in the previous grade. These results are particularly acute in ELA. In several control

groups of ELL student populations, these students scored higher than they did in MCPS.

The trajectory performance of certain subgroups of students in MCPS is of concern. In

Math, 38 percent of 3rd-grade African Americans students in MCPS reach proficiency on

the PARCC. By 6th grade, the percentage of this same subgroup reaching proficiency is

19 percent. In ELA, while proficiency rates in a number of grades has risen over the last

three years, 8th-grade proficiency is flat at 31 percent.

Overall, student performance in literacy is stronger than in mathematics.

6 Specifically, 3rd grade ELA proficiency levels are lower than 4th and 5th grades; 6th grade proficiency levels are lower

than the two years before or after. In addition, 7th grade math proficiency levels are lower than the two preceding years.

However, caution is in order in interpreting the math results, given the far higher percentage of students now moving to

Algebra in earlier grade levels than before.

MCPS: Curriculum 2.0 Evaluation Summary and Recommendations 8

Recommended Next Steps The RFP requested that the research team provide guidance on the policy implications of the findings

and recommendations. Our counsel is based on the experience of other districts engaged in curricular

transitions and on the research base that supports the effective implementation of high-caliber

curricula.

Considerations:

1. There is a strong case for beginning a transition away from Curriculum 2.0 and towards

externally developed evidenced-based researched and reviewed instructional materials.

Several ELA and math curricula rank higher than Curriculum 2.0 on standards-alignment

metrics; many are available as Open Educational Resources (OER materials on the Web are

downloadable for free); and several offer other strong advantages in terms of materials for

struggling or advanced students, as well as other resources for Special Needs and English

Language Learners.

2. There is also a strong case for implementing new materials across several academic years. For

a school district of the size of MCPS, the need to provide multiple levels of stakeholder

engagement, logistical support, assistance for principals and curriculum directors, and above

all extensive up-front and ongoing professional assistance and development for teachers,

creates a major implementation challenge. Failure to adequately plan and execute effectively

on these elements of transition will undermine – to varying but serious degrees – the efficacy

of the transformation. Research on major instructional transitions – from the 1990s onwards

– strongly supports this approach.7 Here is the conclusion of a recent study of Core

Knowledge:8

Multilevel support for change was required for Core Knowledge to be successfully

implemented. Successful implementation relied on instructional leadership from the

principal, teacher willingness to change, and support from the district…Successful

implementation also relied upon…teacher-planning time, as well as the organization

of time, space, and professional development.

7 See for example, Youree, D. G. (1998). The implementation of core curriculum: A case study (Order No. 9907867).

Available from ProQuest Dissertations & Theses Global. (304482675). Retrieved from

https://search.proquest.com/docview/304482675?accountid=11752. 8 Stringfield, S., A. Datnow, G. Borman, and L.T. Rachuba. 2000. “National Evaluation of Core Knowledge Sequence

Implementation: Final Report.” Baltimore, MD: Johns Hopkins University Center for Research on the Education of

Students Placed at Risk. https://eric.ed.gov/?id=ED451282.

MCPS: Curriculum 2.0 Evaluation Summary and Recommendations 9

3. Professional development for teachers on the new materials is critical. Strong research shows

that more than half of the impact of a curriculum adoption is dependent on how teachers

implement the new materials. Additional research shows that the same curriculum,

implemented with greater or lesser fidelity, can range in its impact on student outcomes from

being negative to adding more than an average of twelve months’ worth of learning in a single

academic year.9

Survey data indicate that, in general, teachers’ use of a district approved curriculum varies, with a

sizeable share of teachers indicating less frequent use of the district curriculum and instead relying

on other resources that they create themselves or find online.10 This data makes it clear that to design,

structure and support a genuine, large-scale adoption and use of a new curriculum across a large

district with some 6,000 teachers is a far more demanding task than simply incentivizing a given

curriculum as the “official” set of instructional materials for the district.

9 Steiner, David. 2017. “Curriculum Research: What We Know and Where We Need to Go.” StandardsWork.

https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf. 10 Opfer, V. Darleen, Julia H. Kaufman, and Lindsey E. Thompson. "Implementation of K–12 state standards for

mathematics and English language arts and literacy." RAND Corporation (2016).

Curriculum Review

Montgomery County Board of EducationMarch 22, 2018

Today’s Discussion

• Background and Context

• Curriculum Review, Findings, and Recommendations (Johns Hopkins Institute for Education Policy)

• Learnings and Next Steps for MCPS

2

Today’s Discussion• Dr. Maria Navarro, Chief Academic Officer• Dr. Erick Lang, Associate Superintendent, Office of

Curriculum and Instructional Programs• Mrs. Niki Hazel, Director, Department of

Elementary Curriculum and Districtwide Programs• Mr. Scott Murphy, Director, Department of

Secondary Curriculum and Districtwide Program• Dr. David Steiner, Executive Director, Johns Hopkins

Institute for Education Policy

3

Background and Context

• Board of Education Policy IFA, Curriculum, requires reviews of curriculum content areas on five-year cycles.

• 2017–2018 marks a five-year review period for Elementary English language arts and mathematics curriculum.

4

Background and Context• Previous curriculum revisions (2003–2004, 2009–2010)

• 2010: Maryland adopts Common Core State Standards, known as Maryland College and Career Ready Standards

• In MCPS: Internally developed online platform to house

curriculum and resources Leveraging teachers currently or recently out of the

classroom as writers of curriculum, materials, and assessments

Feedback from stakeholders

5

Curriculum Review• Time for curriculum review• Ongoing feedback from stakeholders• Sense of urgency and continuous improvement• National conversation about internal vs. external

curriculum• Recent development of externally developed high-

quality instructional materials• Rapidly changing technology and assessment

infrastructure6

Curriculum Review• Request for Proposals (RFP) in 2017

• Johns Hopkins Institute for Education Policy

• Review of K─8 curriculum of English language arts and mathematics:o Written Curriculum o Taught Curriculumo Learned Curriculum

• July 2017 through January 2018

7

The Evaluation Team

• Johns Hopkins Institute for Education Policy

• Johns Hopkins Center for Research and Reform in Education (CRRE)

• Student Achievement Partners

• Lengel Educational Consulting

• StandardsWork

Curriculum Matters

Curriculum matters. High-quality research suggests thatusing best-in-class instructional materials can improvestudent learning even more than other, more well-known,interventions such as expanding preschool programs, givingmerit pay to successful teachers, decreasing class sizes, orincreasing the number of charter schools in a district.1

Despite this, few states and districts view the curriculum asan important policy lever for change.

1Matthew Chingos and Grover Whitehurst, “Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core” (Washington, DC: Brookings Institution, April 10, 2012), https://www.brookings.edu/research/choosing-blindly-instructional-materials-teacher-effectiveness-and-the-common-core/., U. Boser, M. Chingos, and C. Straus, “‘The Hidden Value of Curriculum Reform’” (Washington, D.C.: Center for American Progress, 2015), https://cdn.americanprogress.org/wp-content/uploads/2015/10/06111518/CurriculumMatters-report.pdf.

Curriculum 2.0 Review• MCPS has a longstanding history of designing and implementing instructional

materials that challenge students, empower educators, and narrow achievement gaps. Curriculum 2.0 was one of the earliest curricula developed in the Common Core era. The curricular landscape has changed since then; research on the effects of standards is significant; new instructional models have entered the market.

• In Montgomery County, curriculum is routinely reviewed in alignment with Board of Education policy (every five years). Coinciding with the arrival of a new superintendent and after approximately five years of implementation of Curriculum 2.0, an external review of the written, taught, and learned curriculum was conducted.

The team undertook a 360° evaluation of Curriculum 2.0, involving multiple, overlappingperspectives and data collections.

1. A formal alignment evaluation comparing Curriculum 2.0 materials in math and ELA,Kindergarten through 8th-grade, against the multiple requirements of the Maryland statestandards.2. An extensive evaluation of classroom practices, involving on-site classroomobservations; analyses of student work; survey data of the views of multiplestakeholders; and focus groups with teachers, content coaches, and specialists. Thepurpose of this review was to listen to those who worked with Curriculum 2.0 inschools—and to examine how the use of Curriculum 2.0 in MCPS classroomsinfluenced the delivery of standards-based education.3. An evaluation of MCPS 3rd through 8th-grade PARCC scores over the past threeschool years to identify the specific grades, subjects, and student groups whereproficiency levels were outliers in comparison to MCPS students in surrounding gradelevels. The evaluation also compared MCPS PARCC scores to comparison schooldistricts with identifiable sub-groups of similar proportion and demography to those inMCPS.

The evaluation surfaced several positive aspects of Curriculum2.0. Among them:

• Most of the observed lessons, in both ELA and math, drewupon the learning goals from Curriculum 2.0.

• Math classrooms use resources from Curriculum 2.0 themajority of the time.

• The middle school math materials of Curriculum 2.0 approachalignment to the high-level requirements of the MarylandCollege and Career Ready Standards.

The evaluation also surfaced a number of challenges associatedwith Curriculum 2.0 that were confirmed across every domain ofthe inquiry. These challenges are elaborated next.

Formal Alignment Analysis

The instrument of analysis was the InstructionalMaterials Evaluation Tool (IMET), created byStudent Achievement Partners, which isregarded as the national gold-standardinstrument for curricula alignment analysis.EdReports, for instance, draws upon the IMETtool in all of its published curricular evaluations.

Alignment• Based on the materials reviewed, the ELA curriculum in K

through 8th grade must be better aligned to central features of the Maryland College and Career Ready Standards.

• The math curriculum in K through 5th grade must be better aligned to central features of the Maryland College and Career Ready Standards.

• Middle school mathematics 6─8 was rated higher for alignment.

Student Work Samples

The instrument of analysis was an MCPS-appropriate version of the Student Work AnalysisTool that is part of Student AchievementPartners’ Instructional Practice Toolkit (IPT). Intotal, the team reviewed 36 different assignmentsand 530 student work samples in math, and 34assignments and 455 student work samples inELA.

Student Work Samples

• Student work samples did not consistently show mastery of the learning standard.

• Student work samples show that fewer than a third of students master their assignments in either ELA or math, although mastery in mathematics is higher than in ELA.

Focus Group AnalysisThe instrument of analysis was an MCPS-appropriate version of the Common CoreKnowledge and Practice Survey developed byStudent Achievement Partners. In total, theresearch team conducted 52 focus groups andinterviews at 20 MCPS elementary and middleschools with 324 educators—including bothteachers and central staff—collecting 2,441comments.

Focus Groups• Agreement about the curriculum’s strengths and weaknesses.

• Overall, participating teachers do not have a positive view of Curriculum 2.0.

• While most teachers use Curriculum 2.0 every day, they supplement it with additional lessons and assignments from other sources.

• Teachers expressed concern that Curriculum 2.0 does not adequately meet the needs of their special education students or students with limited English proficiency.

Classroom ObservationsThe instrument of analysis was the InstructionalPractice Guide (IPG) developed by StudentAchievement Partners. In total, the researchteam conducted observations in 20 schoolsacross the district, encompassing 14 elementaryschools and 6 middle schools. The teamobserved 77 classrooms for 20–30 minuteseach: 38 ELA classrooms and 39 mathclassrooms.

ObservationsMath:• Difficulty making mathematical content clear in their lessons and making “the

mathematics of the lesson explicit through the use of explanations, representations, tasks, and/or examples.”

• Teachers across all elementary- and middle-school grades make mathematical errors; significant errors occurred in 18% of the classrooms, and minor mathematical errors in 23% of the classrooms.

ELA:• Text complexity• Foundational skill instruction words• Textual evidence

Teacher Survey

The survey questions were derived from the RANDCorporation’s American Teacher Panel study withMCPS-appropriate modifications. In total, 1,852teachers responded to the survey. 56% of respondentswere from elementary schools, 44% from middleschools. The respondents were seasoned educators,having taught an average of 15 years.

Teacher Survey• ELA teachers use non-Curriculum 2.0 materials more frequently than they do

Curriculum 2.0 materials.

• While math teachers use Curriculum 2.0 materials more frequently than do ELA teachers, approximately the same percentage of math teachers (87%) use self-developed materials, and only slightly fewer use materials selected from the Internet, daily or almost daily.

• Teachers do not believe Curriculum 2.0 adequately meets the needs of special education or ELL students.

• Teachers report that they spend time on reading and skill development, but this was not documented in the classroom observations.

• Overall, student performance in literacy is stronger than in mathematics.

• In ELA, while proficiency rates in a number of grades have risen over the last three years, 8th-grade proficiency is flatat 31%.

• Proficiency levels in 3rd- and 6th-grade ELA are lower than those in surrounding grades.

• The trajectory performance of certain subgroups of students in MCPS is of concern. In Math, 38% of 3rd-grade AfricanAmericans students in MCPS reach proficiency on the PARCC. By 6th grade, the percentage of this same subgroupreaching proficiency is 19%.

• Proficiency levels among students with limited English are lower in each grade level than in previous grade.

• Specifically, 3rd-grade ELA proficiency levels are lower than 4th- and 5th- grades; 6th-grade proficiency levels are lowerthan the two years before or after. In addition, 7th-grade math proficiency levels are lower than the two precedingyears. However, caution is in order in interpreting the math results, given the far higher percentage of students nowmoving to Algebra in earlier grade levels than before.

Considerations1. There is a strong case for beginning an immediate transition awayfrom Curriculum 2.0 and towards the highest-quality instructionalmaterials: several ELA and math curricula rank considerably higherthan Curriculum 2.0 on standards-alignment metrics; many areavailable as Open Educational Resources (OER materials on the Webare downloadable for free); and several offer other strong advantagesin terms of materials for struggling or advanced students, as well asother resources for Special Needs and English Language Learners.

Considerations2. There is also a strong case for implementing new materials across severalacademic years. For a school district of the size of MCPS, the need to providemultiple levels of stakeholder engagement, logistical support, assistance forprincipals and curriculum directors, and above all extensive up-front and ongoingprofessional assistance and development for teachers, creates a majorimplementation challenge. Failure to adequately plan and execute effectively onthese elements of transition will undermine—to varying but serious degrees—theefficacy of the transformation. Research on major instructional transitions – from the1990s onwards – strongly supports this approach.9 Here is the conclusion of arecent study of Core Knowledge:10

Multilevel support for change was required for Core Knowledge to be successfully implemented.Successful implementation relied on instructional leadership from the principal, teacher willingness tochange, and support from the district…Successful implementation also relied upon…teacher-planning time, as well as the organization of time, space, and professional development.

9See for example, Youree, D. G. (1998). The implementation of core curriculum: A case study (Order No. 9907867). Available from ProQuest Dissertations & Theses Global. (304482675). Retrieved from https://search.proquest.com/docview/304482675?accountid=11752.10Stringfield, S., A. Datnow, G. Borman, and L.T. Rachuba. 2000. “National Evaluation of Core Knowledge Sequence Implementation: Final Report.” Baltimore, MD: Johns Hopkins University Center for Research on the Education of Students Placed at Risk. https://eric.ed.gov/?id=ED451282.

Considerations3. Professional development for teachers on the new materials iscritical. Strong research shows that more than half of the impact of acurriculum adoption is dependent on how teachers implement thenew materials.11 Additional research shows that the samecurriculum, implemented with greater or lesser fidelity, can range inits impact on student outcomes from being negative to adding morethan an average of twelve months’ worth of learning in a singleacademic year.12

11Joseph A. Taylor et al., “An Efficacy Trial of Research-Based Curriculum Materials With Curriculum-Based Professional Development,” American Educational Research Journal 52, no. 5 (October 2015): 984–1017, https://doi.org/10.3102/0002831215585962.12Steiner, David. 2017. “Curriculum Research: What We Know and Where We Need to Go.”StandardsWork. https://standardswork.org/wp-content/uploads/2017/03/sw-curriculum-research-report-fnl.pdf.

Considerations4. Survey data indicate that, in general, teachers’ use of a sanctionedcurriculum varies, with a sizeable share of teachers indicating thatthey use the sanctioned curriculum infrequently and instead rely onother resources that they create themselves or find online.13 This datamakes it clear that to design, structure, and support a genuine, large-scale adoption and use of a new curriculum across a large district withsome 4,000 teachers is a far more demanding task than simplyincentivizing a given curriculum as the “official” set of instructionalmaterials for the district.

13Opfer, V. Darleen, Julia H. Kaufman, and Lindsey E. Thompson. "Implementation of K–12 state standards for mathematics and English language arts and literacy." RAND Corporation (2016).

RecommendationBased on these considerations, the evaluation team stronglyrecommends that MCPS implement new instructional materials inELA and math. Specifically, we recommend that MCPS—at theearliest possible date—design and implement a multi-year phase-in of newly selected K through 8th-grade ELA and math materialsover a period not greater than the next three academic years. Thistime frame balances the advantages that will accrue to studentsand teachers by providing them with the very best learning toolsnow available, with the milestones that must be met in order torealize the full learning impact of the new materials.

Learnings for MCPS• Ongoing cycle of continuous improvement, value of

external feedback

• Curriculum (i.e., anchor texts, quantitative measures, close reading, etc.)

• Changes to instructional practices

• New models of professional learning

• Communication with stakeholders

• Value-add of highly rated externally available materials

30

Next Steps• Move away from utilizing central services staff and

teachers to write curriculum and assessments.

• Adopt external curriculum and assessments developed by curriculum and assessment experts.

• External evaluations and ratings (i.e., Ed Reports).

• Significant change since the early days of the Common Core when there were few external products of sufficiently high quality.

31

Next Steps• Transition to new curriculum over a multiple year period.

• Implement a robust professional learning plan.

• Begin in 2018–2019 in subset of schools.

• Focus on successful implementation and support to schools.

Elementary: K-5 Math (subset of schools) K-5 Literacy (subset of schools) Science (all schools)

Middle: English 6, Math 6 (subset of schools)32

Next Steps• March: Communication and Request for Proposals (RFP)

• April─May: Stakeholder feedback, benchmarking, expert review, and evaluation

• May: Curriculum identified based on RFP process

• May: Identification of participating schools

• June: Recommendation for Board approval

• July─August: Professional learning

• 2020–2021: Full implementation in all schools33

Discussion

34