DiggingInto

Student Data

Oct. 24, 2006

Plan for Today4:10-4:25 Introduction to Digging Deeper

4:25-4:55 Looking at Student Work

4:55-5:45 Single Problem Assessment Protocol

5:45-5:55 Break

5:55-6:15 A Sample Data Overview

6:15-7:00 School Team Work

The Data Wise Improvement Process

What are we digging for?

GoalTo identify a

learner-centeredproblem

that we want to work together

to solve

Steps of the Process1. Share a Data Overview with faculty

2. Begin to think about how to frame a learner-centered problem

3. Start a conversation about what else we need to know to define the problem more precisely

4. Collect the data and state the problem

Two Approaches to “Digging”

1. Look closely at a single data source

2. Explore multiple data sources

1. Look carefully at a single data source Choose a source

that will help you really understand student thinking

Allow for challenging of assumptions

Why start with a single data source? Help move past “stuck points”

Slow down, avoid leaping to solutions

Make the data analysis more engaging

Move beyond generalizations

0

1

2

3

4

Average Score

4 11 26 27

Question Number

Grade 4 MCAS Math 2006: Our Open Response Scores vs. State

Our School State

Student Work on Question 26

2. Dig into multiple data sources “Triangulate” on

the problem

Get a more complete picture of student performance

Why dig into multiple sources? Develop a shared understanding of the

knowledge and skills students need

Develop a common language

Avoid making inappropriate inferences from test results

Questions for Reflection Do you have a solid understanding of why

students are performing as they are?

Can you state the learner-centered problem in a way that focuses on the knowledge and skills you want students to have?

Is your understanding of the problem supported by multiple sources of data?

If you solve this problem, will it help you meet your larger goals for students?

A306Data Wise: Step-by-Step Guide for Using Assessment Results to Improve Teaching

& Learning

Harvard Graduate School of EducationOctober 24, 2006

Looking at Student

WorkSteve Seidel

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

Thinking of student work as “raw” data…

What to do with that data?

How to make sense of it?

How to use those insights to improve instruction?

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

Questions for a Pair/Share: How have you encountered “looking at

student work?”

What worked and what didn’t work about your experiences in structured conversations about student work?

What questions did those experiences leave you with?

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

Five things you need to use that data well:

Collaboration

Clarity of purpose

A structure or method for analyzing it

A way to integrate that analysis into a larger view of what is going on in your classroom, school, or district

Follow-up and follow-through

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

A Brief History of Looking at Student WorkBefore 1985…

The Prospect Center and Descriptive Review Processes Literacy “Digs” Teacher’s Seminars on Children’s Thinking

1985 to 1990… Portfolios, Process-folios…but where’s the assessment?

1990 to now… The Massachusetts School Reform Act of 1993 Coaches, Coaching, and Looking at Student Work:

Required, then rejected

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

Some lessons learned over the years from looking at student work:

You can gain insights in many realms – about learners, the learning environment, the nature of work on a particular task.

You can’t do it alone. (Or, at least, you are much better off doing it with other people.)

You need a structure that serves your purpose.

Look first (and last) at work of students who you feel are not having satisfactory learning experiences in your classroom or school.

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

How to find the one in the many and the many in the one?

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

9th Grade Student’s Self Assessment

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

The Steps of the Protocol (approximately 35 minutes)1. Read the problem and work out your own answer to it. Keep some notes

on your thinking process. (5 minutes)

2. Share observations about the problem and your own problem-solving process. Specifically, what was striking to you about the problem and your efforts to solve it? (5-7 minutes)

3. Look at the student work (read silently). (3 minutes)

4. Describe what this student did and how she/he thought about the problem. (5 minutes)

5. Identify and share questions that have come up for you about this student’s grasp of the mathematical content of this question, based on your examination of this work so far. (5 minutes)

6. Share ideas about the implications for teaching and learning that you draw from your examination of this work. (10 minutes)

The Single Problem Assessment ProtocolDeveloped for Data Wise □ Steve Seidel

Harvard Graduate School of Education □ ©Steve Seidel □ October 2006

Looking at student work in the context of the Data Wise process

Franklin High School Data Overview: Mathematics

Instructional Leadership Team October 2006 Meeting

Purpose of today’s meeting

To start a conversation about math performance at Franklin by looking at

how our 10th graders performed on the state

test last spring

Agenda

3:00-3:25 Overview of presentation/discussion

3:25-3:45 Breakout to brainstorm questions

3:45-4:00 Reconvene to discuss next steps

How has performance changed over time?

STUDENT PERFORMANCE -- MATHEMATICSGrade 10 State Comprehensive Assessment

Franklin High School, 2003-2006

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2003 (n=430) 2004 (n=422) 2005 (n=425) 2006 (n=418)

Year

Perc

en

tag

e o

f S

tud

en

ts

Advanced

Proficient

Needs Improvement

Failing

How did 10th graders perform in 2006? DISTRIBUTION OF STUDENT

PROFICIENCY-- MATHEMATICS Grade 10 State Comprehensive AssessmentFranklin High School, Spring 2006 (n=418)

47%

39%

10%

4%

0%

10%

20%

30%

40%

50%

60%

Failing Needs Improvement Proficient Advanced

Proficiency Level

Pe

rce

nta

ge

of

Stu

de

nts

Compared to state? DISTRIBUTION OF SCHOOL AND STATE

STUDENT PROFICIENCY-- MATHEMATICSGrade 10 State Comprehensive AssessmentFranklin High School, Spring 2006 (n=418)

47%

39%

10%

4%

34%

27%

15%

24%

0%

10%

20%

30%

40%

50%

60%

Failing Needs Improvement Proficient Advanced

Proficiency Level

Pe

rce

nta

ge

of

Stu

de

nts

School State

Which students are failing? By program… Number of Students in Each Proficiency Groupwith Failing Students by Instructional Program

Grade 10 State Comprehensive AssessmentFranklin High School, Spring 2006 (n=418)

Advanced4%

Proficient10%

Needs Improvement

38%

Failing47%

English Language Learners (n=79)

11%

Students with Disabilities (n=82)

10%

Students in Regular Education

(n=257) 27%

Which students are failing? By race/ethnicity…Breakdown of Failing Students, Race/Ethnicity

Grade 10 State Comprehensive AssessmentFranklin High School, Spring 2006 (n=418)

Advanced4%

Proficient9%

Needs Improvement

40%

Hispanic (n=50) 12%

Asian(n=3) 1%

Other48%

African-American (n=118) 29%

White(n=22) 5%

Mixed/Other (n=2) 0%

Native American(n=1) 0%

How does performance differ by strand?

PERCENTAGE OF STUDENTS ANSWERING EACH MULTIPLE-CHOICE ITEM CORRECTLYGrade 10 State Comprehensive Assessment - Mathematics

Franklin High School, Spring 2006 (n=418)

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

27 29 40 38 25 24 9 12 7 6 11 8 4 1 26 10 34 13 5 39 3 33 2 35 37 36 28 30 23 14 22

Item Number

Perc

en

tag

e o

f S

tud

en

ts

Geometry and Measurement

Number Sense Patterns, Relations, and Functions

Statistics and Probability

Compared to state?PERCENTAGE OF STUDENTS ANSWERING EACH MULTIPLE-CHOICE ITEM CORRECTLY

Grade 10 State Comprehensive Assessment - Mathematics Franklin High School, Spring 2006 (n=418)

-0.40

-0.35

-0.30

-0.25

-0.20

-0.15

-0.10

-0.05

0.00

0.05

0.10

29 40 27 38 25 24 9 12 4 7 11 8 1 6 26 10 13 34 3 5 33 39 2 35 37 36 14 23 28 30 22

Item Number

Dif

fere

nce

in P

erce

nta

ge

of

Stu

den

ts

Geometry and Measurement

Number SensePatterns,

Relations, and Functions

Statistics and Probability

What questions does this overview raise? Brainstorm in groups:

Group 1 Group 2 Group 3 Group 4

Roger Mallory Sasha Adelina

Eddie Will Karyn Dottie

LaShawn Miguel David K. Ervin

David S. Sondra Jamal

What should we do next and who should do it?

For Nov 3: Data Overview

Comprehensive picture of your school’s data

Intended to be shared with your faculty

Opportunity to address adaptive challenges

Opportunity to develop technical skills

Data Overview: Steps 7-11

7. Decide which charts to share

8. Identify additional data

9. Produce charts of this data

10. Plan how to structure discussion

11. Create PowerPoint presentation