copyright © 2008 learning point associates. all rights reserved. surveys of the enacted curriculum...

Copyright © 2008 Learning Point Associates. All rights reserved.

Surveys of the Surveys of the Enacted CurriculumEnacted Curriculum

Cary CuiccioLisa Palacios

March 6, 2009

www.learningpt.org2

Learning Point Associates

Our Vision An education system that works for all

learners.

Our Mission We deliver the knowledge, strategies,

and results to help educators make research-based decisions that produce

sustained school improvement.

www.learningpt.org

Research has found that faculty in successful schools always question

existing instructional practice and do not blame lack of student achievement

on external causes.

— Carl Glickman, 2002

www.learningpt.org

What if?

… there was a powerful tool that compared teacher’s reflections of content instruction to state standards, benchmarks and assessments?

… you could compare how your teachers teach content compared to how others teach across the nation?

… you could have consistency across grade levels on both time teachers spend on content as well as depth of instruction?

www.learningpt.org

What if?

… there was a tool that compared teacher’s reflected instructional practices to other teachers’ across a school, district or the nation?

… you could use data on instructional quality and content to guide professional development?

… you could use anonymous teacher data to start a powerful school discussion about teachers’ curricular and professional development needs based on data and observed differences between student achievement scores and schools curriculum?

www.learningpt.org

Introducing: The SEC

Online web-based surveys in Mathematics, English Language Arts, Social Studies and Science.

Groups data that are reported in user-friendly charts.

Assists in facilitating group discussions for improved instruction across classrooms, schools and districts.

A practical, reliable set of data collection tools used by teachers.

www.learningpt.org7

www.learningpt.org

• State Departments of Education (selected)

• National Organizations (College Board,

ACT, NAEP, WIDA)

• Council of Chief State School Officers

• Wisconsin Center for Education Research

• Learning Point Associates

• TERC

SEC Collaborative

www.learningpt.org

History of the SEC

The Surveys of Enacted Curriculum (SEC) Collaborative Project was initiated in 1998.

Surveys are available K-12 for Mathematics, Science, English Language Arts and Reading, and Social Studies.

www.learningpt.org

History of SEC (cont.)

Over the past ten years the Council of Chief State School Officers (CCSSO) and their partners have worked with states to implement the SEC tools for data collection, analysis, and reporting.

www.learningpt.org

Research on the SEC

• MSP-PD Study (Blank, Smithson, Porter, Nunnaley, Osthoff, 2006)

Use of SEC data for program evaluation

• Reform-Up-Close (Porter, Kirst, Osthoff, Smithson, Schneider, 1993)

Validation of teacher self-report survey data.

• Upgrading Mathematics (Gamoran, Porter, Smithson, White, 1997)

First content analysis of assessment using content language.Predictive validity of alignment index comparing instruction & assessments

• Data on Enacted Curriculum (Blank, Porter, Smithson, 2004)

Use of SEC data to facilitate school improvement effortsFirst content analysis of state standards


The ELA Survey Part One:The ELA Survey Part One:

Instructional Content and Cognitive Demand

www.learningpt.org

Instructional Content

A tool to address a key question:Is classroom instruction aligned with

state content standards and assessments?• Content Emphasis• Cognitive Expectations

www.learningpt.org

Instructional Topics

Topics provide the framework for data collection and comparison.

Topics include the range of topics found in English Language Arts classrooms, K–12.

Stable framework—not intended for a particular state or curriculum; rather with the content area in mind generally.

Neutral matrix—intends to include all possible topics that could be taught.

www.learningpt.org

www.learningpt.org

100 Phonemic awareness

1000 Elements of Presentation (Verbal and Written)

200 Phonics 1100 Writing applications

300 Vocabulary 1200 Language Study

400 Text and print features

1300 Listening and Viewing

500 Fluency 1400 Speaking and Presenting

600 Comprehension 1500 Forms of Text

700 Critical Reasoning 1600 Genre (fiction or non-fiction)

800 Author's craft 1700 Sources of Text

900 Writing processes 1800 Choice

K-12 English Language Arts Reading Content Areas

www.learningpt.org

www.learningpt.org

600 Comprehension

601 Word meaning from context

602 Phrase

603 Sentence

604 Paragraph

605 Main idea(s), key concepts, and sequence(s) of events

606 Descriptive elements (e.g., detail, color, condition)

607 Narrative elements (e.g., events, characters, setting, and plot)

608 Persuasive elements (e.g. propaganda, advertisement, and emotional appeal)

609 Expository or informational elements (e.g., explanation, lists, and organizational patterns such as description, cause-effect, and compare-contrast)

610 Technical elements (e.g., bullets, instruction, form, sidebars, etc.)611 Electronic elements (e.g., hypertext links, animations)

612 Strategies (e.g., activating prior knowledge, questioning; making connections, predictions; inference, imagery, summarization, re-telling)

613 Self-correction strategies (e.g., monitoring, cueing systems, and fix-up)

614 Metacognitive processes (e.g., reflecting about one's thinking)

615 Interpreting maps, graphs, charts

616 Test-taking strategies

690 Other

K-12 English Language Arts Reading Specific Content

www.learningpt.org

Cognitive Expectations

State standards and assessments have inherent cognitive expectations.

Instructional activities also have cognitive expectations.

The SEC measures the cognitive expectations of instruction.

www.learningpt.org

Cognitive Expectations English Language Arts and

ReadingMemorize and RecallPerform Procedures and ExplainGenerate, Create, and DemonstrateAnalyze and InvestigateEvaluate


Data Interpretation Data Interpretation Part One:Part One:

Instructional Content/Cognitive Demand aligned to State

Standards• Contour Maps

• Tile Charts

www.learningpt.org

Survey Instrument: Alignment

Subject Content

Student Specific Topics

Time Spent on Topic

Student Expectations

www.learningpt.org

Reading SEC Contour Maps

MemorizePerform

CommunicateConjecture

Connect0

0.020.040.060.08

0.1

Operations

Number Sense

Measurement

Geometric Concepts

Algebraic Concepts

Data Analysis

Instructional Technology

MemorizePerform

CommunicateConjecture

Connect

State J Grade 8Mathematics Instruction

www.learningpt.org

www.learningpt.org

This data is from 134 Grade 3 teachers.

Alignment describes how well your instruction aligns to standards or assessments. 1.0 is perfect; anything above .5 is fine.

You read this like a topographical map. Darker areas are peaks and represent more time spent on that topic at that cognitive demand level. Here, the 3rd grade teachers are spending more time on phonemic awareness and phonics, and at more levels of cognitive demand, than the standards require.

Same here in terms of topographical maps. Teachers are spending less time on speaking and presenting than the standards require.

This map is the NY state standards specific to your grade. You can also look at the NY state assessments.

The map on this side will be your data, and only you can access it. That’s why this data is from an entire grade.

You can also click on any subtopic and get another map comparing your teaching to standards and/or assessments at a “fine grain” level.

www.learningpt.org


ActivityActivity


The Survey Part Two:The Survey Part Two:

Instructional Practices

www.learningpt.org


A tool to address key questions:

Are instructional practices optimized for student achievement?

Are the professional development efforts aligned with needs and priorities?

Are teachers’ opinions and belief in line with district and school initiatives?

www.learningpt.org


Homework

Classroom assessments

Opinions and beliefs

Classroom Instructional preparation

Instructional influences

Professional development

Personal characteristics

Formal course preparation


Taking the SurveyTaking the Survey

www.learningpt.org

www.seconline.org

www.learningpt.org

Completing the Online Survey

Approximately 60-90 minutes to compete.

High-speed Internet connection required.

May be completed in multiple sittings.

Data are saved as each section is submitted.


Data Interpretation Data Interpretation Part Three:Part Three:

Powerful Discussions

www.learningpt.org

And so you just threw everything together? Matthews, a posse is something you have to ORGANIZE.

www.learningpt.org

What does the SEC show?

Instruction

Assessment

Standards

Intersection of what is taught with what is tested.

Intersection of what is taught with what is in standards.

Intersection of whatis tested with what is in the standards.

Taught,tested, and inthe standards.

www.learningpt.org

Traditional Model for School Improvement

Identify “Best Practices” for Achievement Problems

Identify Shortcomings in Student Performance

www.learningpt.org

Better School Improvement

Identify Appropriate Programmatic Solutions

Identify Shortcomings in Student Performance

Review Instructional Program Data Relative to Concerns

The Surveys of Enacted Curriculum

www.learningpt.org

Applications for SEC

Aligning standards, assessments, and curriculum

Improvement of instruction

Interpreting student assessment results

Needs assessment and program evaluation

Indicators system for monitoring progress

www.learningpt.org

Possible Data Analysis Outcomes

Targeted professional development

Changes in environmental factors

Positive climate of change

Continuous improvement process

Review of curriculum materials

Other data-driven decisions and actions

www.learningpt.org

What SEC isn’t --

Individual Teacher Evaluation• The “wall” of 3

A tool to judge alignment without school/district context and/or discussion with staff.

The only piece of data for measuring alignment of teacher instruction.

www.learningpt.org

“Regardless of what a state policy requires or what a district curriculum spells out,” says Andrew C. Porter of Vanderbilt University, “the classroom teacher ultimately decides” what to teach.

Education Week, October 8, 2003

www.learningpt.org

Think, Pair, Share

Discuss what you’ve learned with the person next to you and make a few notes.• What was the purpose today?• Similarities in your thoughts,

perspectives? • Based on what you read – how could the

SEC help to improve your classroom, school or district?

www.learningpt.org54

Cary Cuiccio

P: 718-483-4279

E-Mail: [email protected]

Lisa Palacios

P: 630-649-6601

E-Mail: [email protected]

copyright © 2008 learning point associates. all rights reserved. surveys of the enacted curriculum...

Documents