using data for targeted interventions

Using Data Effectively in Decision Making

Karen Greenwood Henke Founder

Copyright © 2007 Nimble Press™. All rights reserved.

Workshop Leader

Karen Greenwood Henke

Managing Director, Nimble PressWriter for the CoSN Data-Driven Decision Making Initiative

[email protected]

mailto:[email protected]


Agenda

• Introductions• The context for

DDDM• Case studies • Group exercises• Making it happen


CoSN Vision to Know and DoLaunched in 2003 to help educators use data effectively

A highly-respected source of up-to-date, unbiased information

An easy-to-use mechanism for educating school leaders A nationally-recognized framework for sharing

knowledge www.3d2know.org

Supported by Founding Partners: ETS, IBM and SAS Additional support from: Co-nect, Dell, Pearson Education, Plato

Learning, PowerSchool, SchoolNet, Texas Instruments, and

media partner Scholastic Administr@tor.

http://www.3d2know.org/


The Context for Data-Driven Decision

Making


No Child Left Behind 2002

• Data collection, analysis and reporting mandated by the federal government.

• States required to set standards, assessment tools, and collect reports.

• School districts must carry out testing, and report disaggregated data.

• All schools accountable for progress of all children on high stakes tests.


National Education Technology Plan 2004

Toward a New Golden Age in American Education calls upon states, districts and schools to: Establish a plan to integrate data

systems and ensure interoperability Use administrative and instructional

data to understand relationships Use assessment results to inform

instruction


• Center for Education Policy reports increases in test scores and a narrowing of the achievement gap

• All states have tests and results are publicly available

• Schools are being ranked and rated based on high stakes test scores

Source: “Answering the Question That Matters Most: Has Student Achievement Increased Since No Child Left Behind?” , Center for Education Policy, 2004

Progress in DDDM Adoption

http://www.cep-dc.org/


“I feel pressure as a teacher, more because I want my students to do well. I don’t want them to fall behind.”

“They bombed in this area that we worked so hard on! What does that say about me?”

“My students do not want to fail. I think that they are really nervous. I try to comfort them.”

Source: “Linking Data and Learning – the Grow Network Study,” Center for Children & Technology, 2005

Intense Pressure on Teachers


The current environment is an opportunity to:

• Use data to transform teaching, learning, and administration

• Inform decisions about everything from class schedules to textbook reading levels to professional development budgets

• Provide a rationale for decisions that parents, teachers, taxpayers, and students can understand

Moving Beyond the Mandate


Changing the Mindset

Use data to catch you doing something right, not punish you for doing something wrong.

Montgomery County MD Superintendent Jerry Weast


Data-Driven Decision Making n. 1 process of making choices based on appropriate analysis of relevant information

Why Use Data?

Definition


1. Assess the current and future needs of students.2. Decide what to change.3. Determine if goals are being met.4. Engage in continuous school improvement.5. Identify root causes of problems.6. Align instruction to standards.7. Provide personalized instruction.8. Track professional development.9. Meet accountability provisions of NCLB.

10. Keep constituents informed about progress.

10 Reasons to Bring Data into Decision Making


• Wealth of data but little actual knowledge of what the data mean

• Scattered and antiquated technology infrastructure – separate legacy systems– impossible to link

• Data accessible only through gatekeeper– data analysts

• Inaccurate, late, or cumbersome data

Data-Rich but Information Poor


Knowledge a guide to action

Informationcontext to create

meaning

Data collection and organization


Tech

nolo

gy,

tra

inin

g,

team

work

A Framework for DDDM


“Digital technologies [enabling DDDM]… allow teachers and administrators to track the stuff of daily instructional decision-making in the classroom.”

•performance data•observational data• informal conversations•portfolios of student work•self-assessment and reflection


Beyond Test Data


ReportsData Warehouse

Reporting and analysis services turn data into useful information

Training to use data to make informed decisions

State and Federal reporting

Dissemination to the community (report cards)

SIS

Finance

Assessment

Instruction

Applications

Personalized Instruction

Source: U.S. Department of Education, 2003

Information Management System


Barriers to More Effective DDDM

Grunwald Associates Page 0

Barriers to More Effective Data-Driven Decision MakingLack of training and interoperability are the main barriers to more effective data-driven decisions. A lack of understanding in what to do with the data is a barrier that’s particular to large and medium districts.

Barriers to Data-Driven Decision MakingTo what degree does your district consider the following to be barriers to

a more effective data-driven decision making process? (N=353)(Percent Responding 8, 9, or 10 on 10-point scale) Large (A)

School District Size

Significantly higher

Medium (B) Small (C)

50%

42%

39%

36%

35%

31%

24%

24%

22%

Lack of training

Interoperability - systems that are unable toshare/exchange data

Lack of understanding in what to do with thedata

Absence of clear priorities on what datashould be collected

Failure to collect data in a uniform manner

Outdated technology / legacy systems

Low quality data - inaccurate or incomplete

Timing of data collection

User interface is too complicated tounderstand reports

47%

44%

41%

36%

38%

34%

24%

23%

19%

55%

42%

44%

37%

35%

30%

25%

25%

24%

42%

36%

17%

28%

28%

22%

20%

23%

21%

C

C


Case Studies


Poway Unified School District (CA)

32,754 students 31 schools Suburban San

Diego

All students learning/ whatever it takes


Poway Improvement Process Organize and clean up years of data Design data warehouse based on student learning

objectives Prioritize delivery to emphasize clarity and meaning Provide professional development for frequent tasks Add filters, reports, and tools based on teacher feedback


Poway Information Management 2001: Web-based benchmark

assessments 2002: Total Information Management

system, integrating data silos 2003: Business analytics

software for data warehouse, management and analysis

2005: Data dashboards for district administrators highlight progress


Poway Professional Development 3 year-long teacher training using data to

set goals for classrooms, grade levels, and individual students, training to help build assessments

Focus on using data to support individual student growth and student ownership of the learning process.

Web pages contain teacher-created resources: current ppts, graphic organizers, notebooks and additional resources (www.pusdmap.com)

http://www.pusdmap.com/


Poway Results

“We are trying to present and deliver data that is useful to a teacher as a guide for instruction. Not to tell teachers what to teach, but to help them make decisions about when and how to teach.” Tracy Jones, Data Systems Supervisor


Poway Results: 2002-2005

Academic performance index increased from 813 to 849

Percent proficient on California Standards for English/language arts increased for all grades

More 3DM Case Studies

English Proficiency

0%

10%

20%

30%

40%

50%

60%

70%

80%

Elementary MiddleSchool

High School

http://www.3d2know.org/best_practices.html


Group Exercise


2005 Performance on State Mathematics Test

Category% of students

scoring at proficient level

Number of students tested

4th Grade Hispanic 74.2% 2,583

8th Grade Hispanic 37.7% 2,234


Your Turn1. Identify a problem that you are

currently facing in your school2. Indicate the types of data you will need

to make a decision 3. Identify the technology that you will

plan to use to address this issue 4. Describe the barriers that you

anticipate in implementing the use of data


Report Out

Problems to address Types of data to

collect Technologies applied Barriers


Make It Happen


Knowledge a guide to action

Informationcontext to create

meaning

Data collection and organization


Tech

nolo

gy,

tra

inin

g,

team

work

A Framework for DDDM


Confusion about Role of Data

“I was on the verge of helping a district begin their D3M trek when we came to an abrupt halt… by the district's instructional and administrative leaders.

“IT was fully prepared to [respond to] challenges associated with locating data in their student information system, implementing data analysis processes, and rolling out a user-friendly query interface.

“IT was not able to allay fears that the "right" data had been identified and was available. Further, only a handful of educators understood the larger context of D3M…”

Source: School CIO reader response


Progress Quadrants

High Change

Low Data

High Change

High Data

Low Change

Low Data

Low Change

High DataCu

ltu

re o

f C

han

ge

Data Management Systems


Data Collection and Organization Is a data warehouse in place?

Are the technical and human support tools in place to move data from warehouse to schoolhouse?

Are systems developed to integrate data into instruction?

Are teachers using assessments to measure progress?Are curriculum and assessments aligned to standards?


Putting Data in ContextDo teachers have access to data in an easy-to-use

format soon after assessment?Does the district provide analytical tools and trained

staff to give decision makers confidence in the data and tools?

Are teachers trained to use item analysis to understand student outcomes and instructional effectiveness?


Taking Action with Data

Does the district provide the tools and training to interpret and query data?

Have data teams developed a process for identifying, recommending and implementing intervention based on data?

Do district and school-site change agents support teachers and their use of targeted interventions?


Key Messages

Implementing a successful district data-driven decision making process requires a collaborative team approach

The process is continual and cyclical Technology tools can be effectively

utilized in the process


2 CoSN Leadership Initiatives

Vision to Know and Do: Data driven Decision Making Initiative www.3d2know.org

Using Technology to Raise the Achievement of ALL Students Initiative www.accessibletech4all.org

http://www.3d2know.org/

http://www.accessibletech4all.org/


Professional Development Resources for district level technology

decision makers Based on CoSN Essential Skills Framework Includes:

backgrounder book workshop CD participant workbook

Go to the CoSN Store

https://my.cosn.org/mycosn/store/

https://my.cosn.org/mycosn/store/


Karen Greenwood Henke

Founder, Nimble Press228 Grant Avenue, 6th

FloorSan Francisco, CA 94108

(415) [email protected]

mailto:[email protected]

http://www.nimble-press.com/

using data for targeted interventions

Education

data systems

instructional data

data accessible

data mean

cumbersome data datarich

wealth of data

test data source

mindset use data tocatch