Creative ways to use data: A toolkit for schools

Susan Barrettsbarrett@pbismaryland.org

Objectives

Review why and how to use discipline data Provide examples of how CCPS schools use various

forms of data to monitor the effectiveness of PBIS Highlight and demonstrate templates utilized to share

information with staff and PBS teams Determine what barriers to learning we have Complete an activity to help plan for data-based

decision making

Data

IS NOT: A scary or “four letter”

word Should not intimidate us Just numbers

IS: Powerful when used to

discuss discipline Empowering when used

by school teams Reviewed frequently to

determine areas of strength and weakness

Scenarios

You work at an elementary school with 400 students. Upon reviewing data at the end of the year you find that your school had 20 suspensions.

You work at a high school with 1000 students. You have a total of 100 days of suspension during the school year.

Scenarios

You work in a middle school of 650 students. Last school year there were 100 referrals.

You work at an elementary school of 450 students. Last year there were 800 referrals

What impact does it have? Think about each of the scenarios

Impact

Administrators Teachers Staff Students Parents School Climate Interventions Support Services needed Academic Achievement

Improving Decision-Making

Problem SolutionFrom

To ProblemProblem

SolvingSolution

Information

Why Collect Discipline Data?

Decision making What decisions do you make? What data do you need to make these decisions?

Professional AccountabilityDecisions made with data (information) are more

likely to be (a) implemented, and (b) effective

From primary to precise

Primary statements are vague and leave us with more questions than answers

Precise statements include information about 5 “Wh” questions: What is the problem and how often is it happening? Where is it happening Who is engaging in the behavior? When is the problem most likely to occur? Why is the problem sustaining?

From primary to precise: An example

Primary statement: “There is too much

fighting at our school”

Precise statement There were 30 more ODRs for

aggression on the playground than last year, and these are most likely to occur from 12:00-12:30 during

fifth grade’s recess because there is a large number of students, and the aggression is related to getting access to the new playground equipment. “

From primary to precise: An example

Primary statement: “ODRs during December

were higher than any month”

Precise statement: Minor disrespect and disruption are

increasing and are most likely to occur during the last 15-minutes of our classes when students are engaged in independent seat work. This pattern is most common in 7th and 8th grades, involve many students, and appears to be maintained by work avoidance/escape. Attention may also be a function of the behavior- we’re not sure.

The data are accurate and valid The data are very easy to collect (1% of staff time) Data are presented in picture (graph) format Data are current (no more than 48 hours old) Data are used for decision-making

The data must be available when decisions need to be made (weekly?)

Difference between data needs at a school building versus data needs for a district

The people who collect the data must see the information used for decision-making.

Effective Data Systems

Data Collection

The “Big 5” Average referrals per

day per month Location Problem behavior Student Time

Summarize the “Big 5”

Is there a problem? If no, what will we do to sustain our efforts? If yes, is problem definable or do we need more

information?

Next steps How will we know if it’s working? Where will we review the data?

Steps to Problem-Solving

Define the problem(s) Analyze the data

Define the outcomes and data sources for measuring the outcomes Consider 2-3 options that might work Evaluate each option

Is it safe? Is it doable? Will it work? Which option will give us the smallest change for the biggest

outcome? Choose an option to try Determine the timeframe to evaluate effectiveness Evaluate effectiveness by using the data

Is it worth continuing? Try a different option? Re-define the problem?

Interpreting Office Referral Data: Is there a problem?

Absolute level (depending on size of school) Middle, High Schools (> 1 per day per 100) Elementary Schools (> 1 per day per 250)

Trends Peaks before breaks? Gradual increasing trend across year?

Compare levels to last year Improvement?

What systems are problematic?

Referrals by problem behavior? What problem behaviors are most common?

Referrals by location? Are there specific problem locations?

Referrals by student? Are there many students receiving referrals or only a

small number of students with many referrals? Referrals by time of day?

Are there specific times when problems occur?

Designing Solutions

If many students are making the same mistake it typically is the system that needs to change not the students.

Teach, monitor and reward before relying on punishment.

An example (hallways)

5:1 Ratio of tickets to referrals

Our data tells us that we should be giving 5 positives to each corrective response

How is that measured? Number of coupons versus number of referrals.

Number of RRR Tickets

Quarter K 1 2 3 4 5 Total

One 306 289 278 236 110 193 1412

Two 678 526 423 278 147 191 2243

Overall 984 815 701 514 257 384 3655

Ratio of Tickets: Referrals

0.0

1.0

2.0

3.0

4.0

5.0

6.0

Triangle of Student Referrals

1-5%

5-10% 5-10%

80-90% 80-90%

Intensive, Individual InterventionsIndividual StudentsAssessment-basedHigh Intensity

6+ referrals

Targeted Group InterventionsSome Students (at-risk)High EfficiencyRapid Response

2-5 referrals

Universal InterventionsAll Students Preventive, proactive

0-1 referral

1-5%

07%%

10-15%

03%

80-90%

90%

Intensive, Individual Interventions Individual Students Assessment-based Intense, durable procedures

Targeted Group Interventions Some Students (at-risk) High Efficiency Rapid Response

Universal Interventions All Settings All Students, Preventive, proactive

1-5% Students with 2 or more referrals

Students with 1 referral

Students with 0 referrals

Triangle of Student Referrals:August/September 2005

Triangle of Student Referrals:April 2006

Actual dataTheory

10-15%

4%

80-90%93%

Intensive, Individual InterventionsIndividual StudentsAssessment-basedIntense, durable procedures

Targeted Group InterventionsSome Students (at-risk)High EfficiencyRapid Response

Universal InterventionsAll SettingsAll Students, Preventive, proactive

1-5%Students with 2 or more referrals

Students with 1 referral

Students with 0 referrals

3%

MinutesHoursDays

Cost Benefit Analysis

Number of referrals Q1 and Q2 2006-2007

233

Number of referrals Q1 and Q2 2005-2006

13 30

Staff6300 14175

Average # of minutes administrator needs to

20

45

Average # of minutes student is out of class due to referral

105 236

548

Student

10960

46606300

24660

10485

14175

0

5000

10000

15000

20000

25000

30000

Student Minutes Admin Minutes

Cost-Benefit Analysis

School name

Average # of Average # of minutes staff need to process referral

5

Number of referrals April 2006 61

Average # of minutes student is out of class due to referral

22

Number of referrals November 2005

132

COST/BENEFIT ANALYSIS WORKSHEET

Enter info below

Robert Moton

Elementary School

2640

12201420

660305 355

0

500

1000

1500

2000

2500

3000

Last

Yea

r

This

Yea

r

Tim

e

Re

ga

ine

d

6

3

1

1

0

1

2

3

4

5

6

Last

Ye

ar

This

Ye

ar

Other data to consider

Is our attendance rate improving? Is our achievement data improving?

How many students are on the honor roll? Are state tests scores improving? What is our graduation rate? How many students are taking AP courses?

What else does the data tell you?

Is there a problem on Bus Cafeteria Hallways

If you have been implementing for many years, are you still seeing the same results? Are older students still motivated by the same

incentives?

Next Steps

Comparing academic and behavior data

1-5%

5-10% 5-10%

80-90% 80-90%

Below grade level 6+ referrals

Approaching grade level

2-5 referrals

On or above

grade level

0-1 referral

1-5%

Classroom Performance:

State-Wide Assessment

:Basic

Borderline

Proficient or

Advanced

Discipline:

What is the academic/behavior connection in your school?

What information do you need to answer this question?

What types of data do you currently use? How often? Is it working? What would make it better? What are your goals when you leave to return to

your building?

Templates

Excel data template

Cost-Benefit Analysis Worksheet

Discipline Data: Essential Questions

Staff have questions regarding effective discipline strategies

How do you collect data?What data do you use?What do we do with the data?When do you know you have a problem?How often do you look at your data? How often is discipline data shared with staff?

Discipline Data is collected to answer questions

What information do you already have? Attendance, suspension, office referrals, achievement scores, tardies, timeout/support room referralsWhat are the critical discipline issues in your building? Who, What, How Often, When, Where?

Discipline Data: Essential Questions

Design intervention to target concern

How do you know what invention is needed?How many students contribute to your referrals? Are referrals coming from one grade, classroom, or area?

Measure success

What do we measure?How do we measure "it"?How often do we measure "it"?How do we know when we have success?How do we know when we need to make changes?Who do we share it with?How do we share it?

Resources

www.pbis.org www.swis.org www.pbssurveys.org www.pbismaryland.org

“Without data, you’re just another person with an opinion”- Unknown