survey of enacted curriculum— © 2004 ccsso all rights reserved. surveys of enacted curriculum: an...

Survey of Enacted Curriculum—http://www.seconline.org

© 2004 CCSSO All rights reserved.

Surveys of Enacted Curriculum:An Introduction to Accessing and Understanding SEC Data

S 2

SEC Collaborating Organizations

•Council of Chief State School Officers www.SECsurvey.org

•Wisconsin Center for Education Research www.SEConline.org

•Learning Point Associates/NCREL www.ncrel.org/sec

•TERC Regional Alliance DEC Project www.ra.terc.edu/DEC

S 3

History

• The Survey of Enacted Curriculum Collaborative Project was initiated in 1998 in mathematics and science.

• Over the past six years CCSSO and partners have work with state agencies to implement the SEC tools for data collection, analysis, and reporting.

S 4

• In 2003, working with state specialists, Learning Point Associates and WCER, the collaborative developed an English Language Arts version of the survey.

• Surveys are available K-12 for Mathematics, Science, and English Language Arts and Reading

S 5

Survey Formats

Printed Web-based

S 6

Results Formats

• Online graphic representations

• Printed graphic representations

• Raw data

S 7

The Survey Returns Data to Make Visible:

• Teachers’ Professional Setting

• Instructional Practice

• Content Alignment

S 8

Survey Instruments – Teachers

• Teacher Characteristics– Teacher Readiness– Professional Development

Experiences– Influences on Curriculum– Teacher Beliefs

• Class Description

S 9

Survey Instrument – Practice

• Instructional Practices (Math) – Scales – A– Use of Homework – E– Classroom Activities– F– Problem-Solving– G– Small Group Work – H– Hands-On – I– Use of Assessments – J

S 10

Survey Instrument – Alignment

• Subject Content– Subject Specific Topics– Time Spent on Topic– Level of Cognitive Demand

S 11

Key Education Questions

• What progress have we made in implementing mathematics state standards?

•ANSWER: Content Alignment Results of Survey of Enacted Curriculum

S 12

Key Education Questions (cont’d)• What do we know about our

instructional program? How can we create a baseline of data as an evaluation of our MSP or Reading First program?

• ANSWER: Instructional Practices Results of the SEC

Charts F, G, H, I, J

S 13

Key Education Questions (cont’d)• What do we need to know about our

instructional program to help us understand why our performance is low?

• ANSWER: – Alignment of Enacted Curriculum to

Standards and Assessments– Charts L (PD in Content area), – M (Influences on Instructional

Practices), – N (Teacher Course-Taking), – O & P (Teacher Readiness),– Q (Beliefs)…

S 14

Key Education Questions (cont’d)

• Did our professional development positively impact our instructional program in our MSP or Reading First program?

• ANSWER: – Practice: Charts F,G, H, I, J, K…– Beliefs– Alignment

S 15

AssessmentStandards

Curriculum

Alignment

S 16

Surveys of Enacted Surveys of Enacted CurriculumCurriculum

A neutral content grid

with cognitive demand

The intended curriculum: State content standards—What students should learn

The enacted curriculum: What teachers teach

The learned curriculum: Student outcomes based on school learning

The assessed curriculum: State (and other) assessments—tested learning

Slide: Created By Learning Point Associates, 2003

S 17

The SEC Content TopicsMathematics, Science, English Language Arts

The SEC provides a neutral, research-based language to describe content of English language arts, mathematics, and science.

Slide: Created By Learning Point Associates, 2003

S 18

The SEC Content TopicsMathematics, Science, English Language Arts

Review the list of topicspresented for thecurrent Content Area. For each topic in the listthat is taught to thetarget class, select aradio buttoncorresponding to 1,2, or3 based on thefollowing definitions:

0 = Not covered1 = Less than 1 lesson2 = 1-5 lessons3 = more than 5 lessons

S 19

Capturing the Cognitive Expectations

For each topic selectedfrom the previousscreen set the cognitiveexpectations forstudents for each of 5categories of cognitivedemand, using thefollowing definitions

0 = No emphasis1 = Slight emphasis2 = Moderate emphasis3 = Sustained emphasis

S 20

Conjecture/ Generalize/ ProveRecite basic mathematics facts

Recall mathematics terms & definitions

Write formal or informal proofs

Recognize, generate or create patternsPerform Procedures

Use numbers to count, order, denote

Follow procedures/instructions

Reason inductively or deductively

Organize or display data

Read or produce graphs and tablesExecute geometric constructions

Demonstrate Understanding Solve Non-routine Problems/ of Mathematical Ideas Make Connections

Communicate mathematical ideas

Analyze data, recognize patterns

Explain findings and results from data analysis strategies

Show or explain relationships between models, diagrams, and/or other representations

Apply mathematics in contexts outside of mathematics

Use representations to model mathematical ideas

Synthesize content and ideas from several sources

Develop/explain relationships between concepts

Expectations for Students in Mathematics

Determine the truth of a mathematical pattern or proposition

Apply and adapt a variety of appropriate strategies to solve non-routine problems

Find a mathematical rule to generate a pattern or number sequence

Recall formulas and computational procedures

Solve equations/formulas/routine word problems

Memorize Facts/ Definitions/ Formulas

Do computational procedures or algorithms

Make and investigate mathematical conjectures

Identify faulty arguments or misrepresentations of data

S 21

Expectations for Students in English Language Arts and Reading

S 22

Recite basic science facts Explain concepts

Recall science terms and definitions Observe and explain teacher demonstrations

Recall scientific formula

Organize and display data in tables or charts

Make observations Classify and compare data

Collect and record data Analyze data, recognize patterns

Use appropriate tools Infer from data

Make measurements, do computations Draw conclusions

Execute procedures

Generate questions, make predictions Apply Concepts/Make ConnectionsPlan and design experiments Use and integrate science concepts

Test effects of different variables

Build or revise theory

0 = 0 =

1 = 1 =

2 = 2 =

3 = 3 =

Conduct Investigations/ Perform Procedures

Expectations for Students in Science

Explain procedures and methods of science and inquiry

Memorize Facts/ Definitions/ FormulasCommunicate Understanding of Science Concepts

No emphasis (Not a performance goal for this topic)

Slight coverage (Less than one class/lesson)

Slight emphasis (Less than 25% of time on this topic)

Moderate coverage (One to five classes/lessons)

Moderate emphasis (25% to 33% of time on this topic)

Sustained coverage (More than five classes/lessons)

Sustained emphasis (More than 33% of time on this topic)

Analyze Information

Apply and adapt science information to real-world situations

Apply science ideas outside the context of science

Response Codes Response Codes

Time on Topic Expectations for Students

None (Not Covered)

S 23

S 24

Stop & Do: Investigate Curriculum & Instruction

• Content Alignment Results• Teacher Characteristics• Target Class Description• Instructional Practice• Professional Development• Teacher Beliefs

S 25

Task 1: Interpret Content Alignment Results

1. Scan English Language Arts or Math Content Chart.

2. Note the information that you readily understand.

3. Highlight the pieces that need further information to interpret.

4. Compare with the others in your group - noting similarities and differences.

S 26

Content Alignment: Online Vs.

S 27

1 Specific content

Amount of instructionaltime = colored of tiles.

Content areas –strands(e.g. measurement).

The learning expecta-tion (cognitive demand)–horizontal axis.

Groups being compared.

Colored boxes facilitateComparison of data Between two groups.

1

2

3

4

6

5

2

3

4

5

6

7Alignment Re-centered:0.5041 = the degree ofAlignment between sides

7

How to Read the Chart

S 28

Interpreting Coarse Grain Scales

White = Not Covered<2.5 % = Between 4 & 5 class periods<5.0 % = About 9 class periods<7.5 % = 13 to 14 class periods>=7.5 % = More than 14 class periods

Interval is 2.5

S 29

Creating Comparisons at Topic Level

Click on the white boxes at the left of thetopic strand.

S 30

Sub-topic Level – Fine Grain

S 31

Topic Level: Fine Grain Scale Conversions

White = Not Covered<0.5 % = 1 class periods<1.0 % = 2 class periods<1.5 % = 3 class periods>1.5 % = 4 class periods

Interval is .5

S 32

Stop & Do: Prepare to Investigate Curriculum & Instruction

• Content Alignment Results• Teacher Characteristics• Target Class Description• Instructional Practice• Professional Development• Teacher Beliefs

S 33

S 34

Floating Bar Charts - Details

• Two columns supports comparing two cohort groups of result.

• Scale: percentage of class time• Standard deviation and the mean

(represents roughly 2/3 of the respondents)

• (n): no results if n <3• No difference between online and

printed survey format for results.

S 35

Task 2: Interpreting Floating Bar Charts

1. Scan Chart F-1: Instructional Activities by grade level.

2. Note the information that you readily understand.

3. Highlight the pieces that need further information to interpret.

4. Compare with the others in your group - noting similarities and differences in highlighted items.

S 36

S 37

41

3

2

S 38

3

Two columns supports comparing two cohort groups of result.Scale: percentage of class timeStandard deviation and the mean (represents roughly 2/3 of the respondents)(n): no results if n <3No difference between online and printed survey format for results.

2

1

4

3

5

How to Read the Chart

S 39

A teacher’s own results.

S 40

Current SEC Disaggregations

• Grade Level• Level of Achievement• Amount of Professional

Development• Percentage of minority students• By class size• By percentage of LEP students• By percentage of Female students

S 41

To access the Report Generator

Go to: www.seconline.org

S 42

To complete a survey, Login as ‘Not Registered?’.

Click on the arrows by ‘Please Select’ to scroll to your project name.

Click ‘Register’.

OHproject

10000

S 43

S 44

Click on the ‘Data Review’ tab.

S 45

Using the Administrator Report Generator

S 46

Using the Administrator Report Generator

• Select content area.• Select the chart you want to use.• Select groups (what cohorts do you want to

compare?).• For example:

S 47

ProcessingYour results are being

processed.Thank you for your patience

S 48

S 49

Budget Planning to Use the SEC

• Print Version of the Survey = $20.00 per person.

• Online Version of the Survey = $10.00 per person

• Membership in the SEC Collaborative (State, District = $15,000 per year) Includes use of surveys, professional development and technical assistance + opportunity to work with other SEC project leaders.

S 50

To Set-Up Your Group To Use the SEC

Call Alissa Minor at WCER: 608 265-0447.

Call CCSSO, 202 336-7044 or your collaborator partner organization.

To Set-Up Professional Development and/or Technical

Assistance for Your Group

S 51

• You've carefully thought out all the angles.

• You've done it a thousand times.

• It comes naturally to you.

• You know what you're doing, its what you've been trained to do your whole life.

• Nothing could possibly go wrong, right ?

S 52

Think Again!