data informed decision-making

Learning-Centered Leadership Development Program for Practicing and Aspiring PrincipalsWestern Michigan UniversityKalamazoo, MI 49008

A Project funded by the United States Department of Education (USDOE), Washing, DC: 2010

Module 1: Data-Informed Decision-Making

2

INTRODUCTION

• Do you Believe in me? - Dalton Sherman• Reflections• Learning Objectives

3

Dalton Sherman

http://www.youtube.com/watch?v=HAMLOnSNwzA

http://www.youtube.com/watch?v=HAMLOnSNwzA

4

Reflection Upon Dalton Sherman’s Speech• Do staff in your school believe that all students can learn?• What does this belief look like in your school?• How do you know that all students are learning?• What changes do you need to make to align practices with

beliefs?

5

Learning ObjectivesAs a consequence of participating in this module, participants will:• Understand and experience the importance of data in a continuous

improvement cycle;• Utilize a data mining tool D4SS (Data for Student Success, MDE) that will

equip you with an understanding about how to disaggregate student data and identify learning gaps in students performance by gender, ethnicity, SES, and learning impairments;

• Learn from other practicing and aspiring school leaders about the effect and challenges of their evidence-based instructional initiatives; and

• Develop and implement a renewal activity in a high priority content area that is designed to improve student achievement.

6

Conceptual Framework for Date-Informed Decision-Making

• What is Data?• Putting your Fear on the Table Regarding the Use of Data• Conceptual Model for Data Use• Collaborative Inquiry Process

7

What is Data?

8

Examples of Data

Numbers Opinions Observations Essays Science projects Demonstrations and …….

Data Can Answer These Questions

1. How are we doing?

2. Are we serving all students well?

3. In what areas must we improve?

Other Questions?

9

10

Principals’ Perception Regarding the Use of Data

Of Teachers

Teachers uncomfortable with data

Teachers cannot read data Data has meaning to classroom Do not know what to do with

data Data not part of teacher training Lack of knowledge data-

instruction No data link to teaching

practices

Of Themselves

Do not understand data use What data do you use Teacher collection of data Need systematic disaggregation Find better assessment tools PD for teachers and principals

11

Principals’ Perception of Time Constraints to Analyze Data

Time to complete tasks Data vs. classroom duties Limited instructional time Time to analyze data Time for collaboration

Time to monitor teacher use Time in getting test results Time in getting data back A year behind-results Holistic approach in working

with teachers

12

Principals’ Perception of Teacher and Student Issues

Teacher cooperation in assessment

Teacher- team cynicism Teachers see data as important Teacher-staff cooperation in data

assessment Inconsistent teacher collection of

student data Student do not take testing

seriously A few teachers see testing as a fad Utility of data

No relevance to individual

students

Results do not reflect current

students

Needs to make sense

Students mirror teacher attitude

Too much student testing

Teacher buying into data

Unsure if data use beneficial

Quality of instruction

No consistence in teacher use of

tools

Put Your Fears on the Table

No,I don’t see

any problem with the data!

What concerns you most about using data to make school decisions?

Internal?

External?

Do the following concerns sound familiar? 13

“Putting data on the table will damage union

negotiations.”

Fear of Data

“My questions about data will sound silly.”

“Will we get sued if we look at student data? What

about privacy issues?”

“Can we trust the data? What if the

numbers are ‘cooked’?”

“If people know the truth about how our

district is doing, we’ll get pummeled.”

14

“I don’t understand the

data.”

“People will take the data out of context to

further their own agendas.” ?

Take Away Your Fear

You don’t have to be a statistician Data are actionable Data must be viewed in relationship to something else Data should be used to establish a focus of inquiry

15

16

School Processes

Description ofSchool Programsand Processes

PerceptionsPerceptions of

Learning Environment

Values and BeliefsAttitudes

Observations

Enrollment, Attendance,Drop-Out Rate

Ethnicity, Gender,Grade Level

Demographics

Standardized Tests

Norm/Criterion-Referenced Tests

Teacher Observations of Abilities

Authentic AssessmentsStudentLearning

Multiple Measures of Data

Demographics Perceptions Student Learning

Demographics- Gender- Grade- Teacher- Age- Time in Building- Behavior- Attendance- Poverty Level- Racial/ethnic- Socioeconomic- Single Parent- Siblings in household- Free/Reduced Lunch

Parental Involvement- Preparedness- Transience-Out of school experiences

Community Support- Programs e.g., Head Start- Services e.g, FIA

Opportunity to Learn- Current Offerings- Extra Curricular Activities

Teacher quality- Qualifications & Credentials- Instructional Practices- Professional Development- Collective Efficacy - Learning Communities- Professional Affiliations

Leadership- Vision, Mission, Goals- Staff Engagement & Perceptions- Parent Engagement & Perceptions- Supervision Practices- Professional Affiliations

Resource Allocation- Budget Allocation- Staffing Patterns - Professional Development- Facility Usage/Maintenance- Technology Distribution

Results Data (Static Data)- MEAP/MME- ACT- AP Testing- District Benchmark Assessments- Standardized Assessments- Graduation Rate- Postgraduate Follow-up

Process Data (Real-Time Data)- Instructional Strategies- Classroom Assessments- Instructional Time on Task- Behavioral Referrals- Books- Writing Samples- Homework Assigned/Completed- Positive Parent Contacts

School Processes

Perception Data- Student Engagement- Student motivation- Student perceptions of success- Values- Beliefs- Culture- Attitudes- Observations

Data Streams Examples

17

1. My school has a written vision that focuses on student achievement.

❏ Yes No I Don’t Know❏ ❏

6. My school is willing to explore ways to use data to measure progress.


2. My school has a general awareness about why data are significant.


7. Everything my school does aligns with our vision.


3. My school has a mission statement that reflects core values and beliefs.


8. My school knows that staffs role is using data to improve student achievement.


4. My school agrees data shows evidence of progress in achieving student goals.


9. My school uses data to set goals? ❏ Yes No I Don’t Know❏ ❏

5. My school has stated, measurable goals that are tied to our vision.


10. My school make decisions based on data based research?


Are We Ready To Use Data More Effectively?

18

19

Types of data

• Input• Process• Outcome• Satisfaction *Set and assess progress toward goals

*Address individual or group needs*Evaluate effectiveness of practices*Assess whether client needs are being met*Reallocate resources in reaction to outcomes*Enhance processes to improve outcomes

Information Actionable knowledge

District School Classroom

SOURCE: Marsh, J. A., Pane, J. F., and Hamilton, L. S. (2006). Making sense of data-driven decision making. Rand Corporation. p. 3.

Conceptual Framework for Data Use

20

Duh!!!

School

Improvement

21

Establishing the Bridge for Student Improvement

Collaborative Inquiry

22

Enabling Collaborative Work

• Schools have an abundance of data. There is the propensity of school officials to relegate the technical work in organizing data to a small group of individual – i e., principal, principal and select teachers, or data specialist.

• This responsibility needs to be shared among all teachers, and ideally, among all members of the school community.

• It is quite apparent that when people are involved in analyzing and interpreting data collaboratively, they become more invested in the school improvement efforts that are generated out of those discussions.

• The more people involved in data analysis and interpretation, the more effective the resulting school improvement efforts will be.

23

John Dewey

What the best and wisest parent wants for his own

child, that must the community want for all of

our children.

24

It’s Easy to Get Lost in the Numbers

63542

63542

37620

609155629098625

8762987620

980098

89365and forget that the numbers represent the hope and future of real children with strengths as well as challenges,

each deserving the kind of education we want for our very own children

Bridging the Data Gap

Imagine two shores with an river in between. On one shore are data—the masses of data now

overwhelming schools:

On the other shore are the aspiration, intention, moral assurance, and directive to improve student learning and close repetitive achievement gaps.

course-taking patterns attendance data survey data and on and on graduation rates

state test data sliced and diced local assessments demographic data dropout rates

25

How do we get there?

26

I saw this new reading program at

the State conference, let’s

try it, it can’t hurt!

If we put more resources into

“Bubble Kids” our scores will

improve

It is evident that those kids cannot learn as efficiently

as others

ResultsData

? ? ?

27

ResultsData

Build and identify the parts of a bridge that is needed to get Data to the Results?

Collaborative Inquiry Is The Bridge

• Schools know that they have to improve• But they often do not know how to improve• Collaborative inquiry is the how• As collaborative inquiry grows, schools shift away from

traditional data practices and toward those that build a high-performing culture of data use

• When engaged in collaborative inquiry, Data Teams investigate the current status of student learning and instructional practice and search for successes to celebrate and amplify.

28

29

Setting the Stage for Collaborative Inquiry

Participants’ Activity:• In this particular activity, participants will discuss the type of

external and internal data they use in their schools. After this participants will identify trends associated with these administrations.

• On post- it notes, participants will be asked to make the following observations and report out in groups the following questions:

30

Group Activity

Question:

1. Is there one particular data type (external or internal) used more often than the other? If so, why?

2. What decisions are made by the use of external and internal data tools? Who make these decisions (by data type)?

3. To what extent is there a close relationship between the gaps in student learning, as identified by the data types, and the initiatives that were developed?

4. What challenges are you facing implementing the initiative in your school?

5. To what extent is the initiative producing the intended results you originally sought? How do you know? What data are you using? If you are not getting the desired results, what are you doing about it?

Demographics Perceptions Student Learning

Demographics- Gender- Grade- Teacher- Age- Time in Building- Behavior- Attendance- Poverty Level- Racial/ethnic- Socioeconomic- Single Parent- Siblings in household- Free/Reduced Lunch

Parental Involvement- Preparedness- Transience-Out of school experiences

Community Support- Programs e.g., Head Start- Services e.g, FIA

Opportunity to Learn- Current Offerings- Extra Curricular Activities

Teacher quality- Qualifications & Credentials- Instructional Practices- Professional Development- Collective Efficacy - Learning Communities- Professional Affiliations

Leadership- Vision, Mission, Goals- Staff Engagement & Perceptions- Parent Engagement & Perceptions- Supervision Practices- Professional Affiliations

Resource Allocation- Budget Allocation- Staffing Patterns - Professional Development- Facility Usage/Maintenance- Technology Distribution

Results Data (Static Data)- MEAP/MME- ACT- AP Testing- District Benchmark Assessments- Standardized Assessments- Graduation Rate- Postgraduate Follow-up

Process Data (Real-Time Data)- Instructional Strategies- Classroom Assessments- Instructional Time on Task- Behavioral Referrals- Books- Writing Samples- Homework Assigned/Completed- Positive Parent Contacts

School Processes

Perception Data- Student Engagement- Student motivation- Student perceptions of success- Values- Beliefs- Culture- Attitudes- Observations

Data Streams Examples

31

32

BREAK

15 Minutes

33

Creating A Data Team

A data team is a team that meets regularly to analyze data and make educational decisions to improve student achievement.

THE DATA TEAM PROCESS1. Collect and Chart

Data

2. Analyze Data and Prioritize

Needs

3. Establish SMART

Goals

4.Select Instructional

Strategies

5. Determine Results

Indicators

Source: Allison, E. et al.. (2010). Data teams. Lead + Learn Press.: Englewood, CO.

6. Monitor and

Evaluate Results

Data Teams

The data team members must:• Be seen as leaders.• Be willing to learn about data in depth.• Must have skills in collaboration,

communication, and leadership.

The functions of the data team are:• To develop expertise on data.• To share data information with staff members

of the schools.• To assist is setting up support systems at the

schools.• To create and complete action plans based on

the data.

The data team members need:• Information about systems to support data

based decision making.• Training in the problem solving process.

Data team members should be expected to:• Meet regularly as a team to develop a plan to

establish using data to improve student performance.

• Have conversations about student achievement.

• Show examples of successful schools.• Set up a system that supports the collection

and use of student data.• Help staff members understand how to use

student data to guide decision making.• Work to secure commitment from staff

35

36

D4SS

Data Analysis Activity

http://www.data4ss.org

User Name: demo_test1Pass Word: fall_01

Data Narrative Statements Criteria

Data Narrative Statements are objective statements of FACT about the school data

They:

1. Represent student achievement, demographics, school programs, school processes, and stakeholder perceptions

2. Communicate a SINGLE idea

3. Are clear and concise – written in sentences or phrases

4. Describe the data; they do not evaluate the data!

5. MUST stand alone; they do not require the data source to accompany them in order to be understandable.

37

Data Narrative StatementsDo they meet criteria from Previous Slide?

Narrative Statement 1 2 3 4 5

1- Spring 2010 Math Assessment shows that our girls do slightly better than the boys.

2- The Spring 2010 Math Assessment shows that 20.5% of our 11th grades students were proficient and 79.5% were not.

3- The Spring 2010 Math Assessment shows that we really need a new math series.

4- In 2009-2010 21.4 % of all our students taking the Math Assessment are proficient; while 20.5% of our 11th graders are proficient and 33.3% of our 12th graders are proficient.

5- Parents do not like the math program.

38

Year: _________ Building: _________________________

Which grade level(s) is not meeting the criteria for grade level proficiency?

What do we need to know more about?

%Proficient

%Proficient

%Proficient

AYP Target (see slide 47

for AYP Target)

%Proficient

AYP Target(see slide 47

for AYP Target)

Content Area Reading Writing Total ELA ELA Math Math

Overall Building – All

Students

Grade 3

Grade 4

Grade 5

Grade 6

Grade 7

Grade 8

39

MEAPBuilding: Content Analysis

MEAPOverall Building Sub-Group Level Achievement Analysis

GradeNumber

of Students

% Proficient

Number of

Students%

ProficientNumber

of Students

% Proficient

AYP Target(see

slide 47 for AYP Target)

Number of

Students%

Proficient

AYP Target(see slide 47 for AYP

Target)

Content Area Reading Reading Writing Writing Total ELA Total ELA ELA Math Math MathAmerican Indian or Alaska Native

Black or African American

Hispanic or Latino

White Asian American, Native Hawaiian or other Pacific

Islander

Multiracial

Economically Disadvantaged

Students with Disabilities

Limited English Proficient

Non AYP-Migrant

40

MEAPSub-Group Analysis

Sub-Group Level Achievement: Choose sub-group for analysis

Year ___________ Group: ____________________

Grade Number of Students

% Proficient

Number of Students

% Proficient

Number of Students

% Proficient

AYP Target (see


Number of Students

% Proficient

AYP Target(see


Content Area Reading Reading Writing Writing Total ELA Total ELA ELA Math Math Math

Building

3

4

5

6

7

8

41

Date: _______________________________

Building: ____________________________

Data Team Members: __________________________________________________________________________________

________________________________________________________________________________________________________________

42

Narrative Statement:

MEAPData Narrative Statements for Sub-Group Analysis

Year: _________ Building: _________________________

MMEBuilding: Content Area Proficiency

In which subject area(s) is your building not meeting the criteria for proficiency?

What do you need to know more about?

%Proficient

%Proficient

%Proficient

AYP Target(see slide 47

for AYP Target)

%Proficient

AYP Target (see slide 47

for AYP Target)

Content Area Reading Writing Total ELA ELA Math Math

Overall Building – All Students

Grade 11

Grade 12

43

MMEOverall Building Sub-Group Level Achievement Analysis

GradeNumber

of Students

% Proficient

Number of

Students%

ProficientNumber

of Students

% Proficient

AYP Target(see


Number of

Students%

Proficient

AYP Target(see slide 47 for AYP

Target)Content Area Reading Reading Writing Writing Total ELA Total ELA ELA Math Math Math

American Indian or Alaska Native

Black or African American

Hispanic or Latino

White Asian American, Native Hawaiian or other Pacific

Islander

Multiracial Economically

Disadvantaged

Students with Disabilities

Limited English Proficient

Non AYP-Migrant

44

MME Sub-Group Analysis

Sub-Group Level Achievement: Choose sub-group for analysis

Year ___________ Group: ____________________

Grade Number of Students

% Proficient

Number of Students

% Proficient

Number of Students

% Proficient

AYP Target(see


Number of Students

% Proficient

AYP Target (see


Content Area

Reading Reading Writing Writing Total ELA Total ELA ELA Math Math Math

Building

45

Date: _______________________________

Building: ____________________________

Data Team Members: ___________________________________________________________________________________

________________________________________________________________________________________________________________

46

Narrative Statement:

MMEData Narrative Statements for Sub-Group Analysis

Michigan Annual AYP Objectives

47

Reflections On Today’s Session

1. What do you remember from today's session (scenes, events, and conversations)?

2. What words are still ringing in your ears?

3. What image captures for you the emotional tone of today's session?

4. What is a key insight from today's session?

5. What name would you call today's session? (Try a poetic title that captures your responses.)

48

49

End

Session 1

50

ResultsData

ContinuousImprovementData UseCollaborationLeadership

Capacity

TrustCultureEquity

51

PANEL DISCUSSION WITH DATA EXPERTS

Kathryn Parker Boudett

http://bcove.me/v4ccqy6q

Joel Klein

http://bcove.me/byt742or

Rudy Crew

http://bcove.me/emn3jz77

Martha Greenway

http://bcove.me/4cf0bkzq

Aimee Guidera

http://bcove.me/trsldt2h

Dan Katzir

http://bcove.me/gy3wbiac

http://www.edweek.org/ew/section/video-galleries/april10-event-data.html

http://bcove.me/v4ccqy6q

http://bcove.me/byt742or

http://bcove.me/emn3jz77

http://bcove.me/4cf0bkzq

http://bcove.me/trsldt2h

http://bcove.me/gy3wbiac

http://www.edweek.org/ew/section/video-galleries/april10-event-data.html

Sm

art

Go

als

Data Teams

Summative Assessments

Perceptual

Demographic

Achievement

Formative Assessments

Ali

gn

me

nt

Qu

estion

s and

Inq

uiry

Process

Revised Instructional Strategies

Revised Instructional Strategies

DataTeam

s

Data Feedback Model

Dis

tric

t W

ritt

en a

nd

Ta

ug

ht

Cu

rric

ulu

m

Dat

a In

ters

ecti

on

A

nal

ysis

data informed decision-making

Education

data data

data time

data teachers

data unsure

data link

importance of data

teacher collection of

data mining tool d4ss