BoQ Critical Element: Data Entry & Analysis Plan

Established

Critical Element D: Data Entry & Analysis Plan

Established13. Data system to collect and analyze ODR data14. Additional data collected (attendance, grades, faculty

attendance, surveys)15. Data analyzed monthly (minimum)16. Data shared with team and faculty monthly (minimum) Data are regularly shared with families (at least monthly) Data shared is monitored for confidentiality Data gathered is disaggregated by race and ethnicity

BoQ Scores by ElementAll Colorado PBIS Schools

*as of 5/30/14

Effective Procedures forDealing with Discipline

Data System Definitions

Referral Form Referral Process

What is an Office Discipline Referral

(ODR)? What it IS:

Kid - Staff Member - Administrator interaction

Underestimation of actual behavior

Piece of information used to make decisions

Data point

What is an Office Discipline Referral

(ODR)?What it IS NOT:

Punishment

A reflection on teacher’s skills

A way to change or re-teach behavior

A first attempt at correcting behavior

Why use ODRs in a PBIS school?

Simply too cumbersome to collect all positive recognitions if given

in the 5:1 ratio!

Characteristics of an Effective Referral Form

The following categories must be included on the form:

Student’s NameDateTime of IncidentStudent’s Grade LevelReferring StaffLocation of Incident

Problem BehaviorPossible MotivationOthers InvolvedAdministrative DecisionOther Comments

Office Discipline Referral (ODR) Forms

In formatting the referral form, you must make sure to answer the following questions:

Who WhyWhat WhenWhere

Clarity on the referral form takes the guess work out of the data entry person’s job

Data will be more reliable and accurate as judgment calls are minimized

Some Examples…

Effective Procedures forDealing with Discipline

Data System Definitions

Referral Form Referral Process

Reasons to Collect and Use Data

Essential for good decision-making

Professional accountability

Decisions made with accurate dataare more likely to be:

(a) implemented(b) effective

Improving Decision Making with Problem Solving

Step 1—Define the problemWhat is the problem?

Step 2—Problem AnalysisWhy is it occurring?

Step 3—Plan ImplementationWhat are we going to do about it?

Step 4—Evaluate Is it working?

CO MTSS Problem Solving Process

Fidelity of Implementation• Benchmarks of Quality (BoQ)• School-wide Evaluation Tool (SET)• Self-Assessment Survey• Team Implementation Checklist

Progress Monitoring• Office Discipline Referrals (ODR)• Suspension/Expulsion• Attendance• Staff/Parent/Student surveys

Two Primary Forms of Data:

Outcomes:Is what we’re

doing making a difference and

benefiting students?

Systems:Are we doing what we said

we’d do?

Progress Monitoring for Behavior

Identify and use meaningful and efficient data sources

Determine trends and patterns

Implement interventions to identified need

Evaluate effectiveness of interventions

On-going

Considerations to Support Data-based Decision-makingDetermine efficient collection procedures

Gathered continuouslyEmbedded as part of school cycle, not something “extra”

Easily accessible

Readable displays

Regular reviewThe people who collect and summarize the data see it is being

used

Systems for Analyzing Data

Utilize an established protocol for analyzing data: Have clear decision rules for determining needs across all

three Tiers

Review monthly with administrators and PBIS team

Share with staff at least quarterly (monthly is best)

What Data to Collect?

Use what you have…

•Office discipline referrals/detentions•Suspensions/expulsions•Referrals by student behavior, staff behavior, and

administrative context•Office referrals per day per month•Climate surveys•Attendance•Referrals to team responsible for problem solving around

students needing more targeted and intensive support •Referrals to special education programs

When to Enter Data

• Daily recommended, but at least weekly (at a minimum)

• With a school-wide data collection system, schools are better prepared to respond proactively to situations

• Schools can begin to identify problems and generate solutions

• Monitor and evaluate success of interventions

When to Analyze Data

Weekly or monthly (minimum)

Allows the team to see if specific interventions are working

Some data on particular students are reviewed more frequently for counseling and family partnering

Reviewing specific behavior incidents more frequently provides further clues regarding effective interventions

How to Analyze Data

How many referrals are there:per day each month?based on location?based on the type of behavior?by student?by time of day?by student motivation?by ethnicity?

What is the range of consequences provided based on the type of behavior exhibited?

“The Big 5”

Questions to Guide Data Analysis

Every time the PBIS team reviews the “BIG 5” graphs, ask these questions…

1. What is going well?2. Do we have a problem? If so, what is it?3. Where? When? How often? Who?4. What other information do we need?5. What are our best guesses about what is happening?6. What is our plan or intervention?7. When and how will we evaluate the effectiveness?

Meaningful Data

Can your current data system calculate the following……

Average Referrals Per Day Per Month

Referrals by Location

Referrals by Time of Day

Referrals by Problem Behavior

Referrals by Student

Referrals by Day of the Week

Referrals by Grade

32

Referrals by Motivation

33

SWIS – School-wide Information System

Tracks discipline referrals for instructional decision making

Provides information for school-wide planningProvides information for targeted group and/or

individual student needswww.swis.org

A Brief Note about SWIS…

Using Data within Multi-tiered System of Supports

Function Literacy/Reading Behavior Numeracy/Math

Universal Screening

DIBELS/ EasyCBM

SWIS; SSBD; SSRS

Under Development

Progress Monitoring

DIBELS/ EasyCBM

SWIS; CICO-SWIS; ISIS-

SWIS

Under Development

Standardized Assessment and Accountability

State Assessment

Measures

State Assessment

Measures(Suspension/ Expulsion

Data, Graduation Rates, etc.)

State Assessment

Measures

SWIS Facilitator Training 2012

How does my school get SWIS?

Identify one SWIS facilitator per district

SWIS facilitator training in Denver (Nov.)

Talk to your regional TAC

Complete SWIS Readiness Checklist

Cost = $250 per school per year

SWISHarassment/Bullying

Data collection to specify Harassment/ Bullying type

Supports data-based decision making and intervention planning around Harassment/ Bullying School-wide Individual Students

Example categories:Racial SexualGender/Sexual OrientationAbilityCyber IntimidationHazingOther

Gibb, A.C. & Skiba, R. (2008). Using Data to Address Equity Issues in Special Education. Bloomington, IN: Center for Evaluation & Education Policy.

“ Data are a critical first step in addressing sources of inequity in education: in order to solve a problem, we must first be able to describe it.”

SWISEthnicity Reports

High priority issue SpEd specifically

“Big 7” – adding Motivation & Ethnicity reports when working with teams

Ethnicity reports within SWIS allow us to progress monitor for prevention

CO PBIS Initiative will be offering webinars specifically around understanding Ethnicity reports

In addition to discipline data….

Look at % of students by ethnicity … in AP & Honors courses on Honor Rolls with passing grades in higher level math, setting course for MS & HS higher math

and calculus Attendance & Tardy rates In SPED

Helpful Data

Mini-SET Wheeling DataBecome familiar with School View (on CDE website)Evaluate overall academic performance and

disaggregated by race & ethnicityKnow your school district’s graduation rates and also

know disaggregated ratesWatch trends from year to year

”The Law of the Scoreboard”John Maxwell

The scoreboard is essential to understanding. It provides a snapshot of the game at any given time.

The scoreboard is essential to evaluating.

The scoreboard is essential to decision making.

The scoreboard is essential to adjusting.

The scoreboard is essential to winning.

Team Time

Determine effectiveness of current data system and process

Review SWIS Readiness Checklist and determine what needs to be accomplished

Determine how data will be shared with staff and families (look at your Mini-SET!)

Complete action plan section (Data Entry & Analysis) identifying goals and tasks to be completed

Outcomes

Team Time

Review your current Office Discipline Referral Form and check for compatibility with SWIS

Revise or create form to include SWIS-required categories

Determine how to present this discipline referral form to the staff

Complete action plan section (Discipline Procedures) identifying goals and tasks to be completed

Outcomes

What is the current data system used by your team? (ex. Infinite Campus, SASI, SWIS, Powerschool)

Can the data system be used to assist your team in making data-based decisions?

How often does your team look at the data?

What data does your team look at?

Do you think that the team understands how to use data to make decisions?

Can you access data disaggregated by race and ethnicity?

Is your discipline data an accurate reflection of what is happening across campus?

Team Time

Reflection Questions

Can your Data System answer the following questions:

how many referrals are there per day each month? how many referrals are there a based on location?

how many referrals are there based on type of

behavior? how many referrals are there by individual

student? how many referrals are there based on time of

day? how many referrals are there based on motivation?

What needs to change in your data system, entry or output to help your team make decisions?

Team Time

Reflection Questions

The contents of this training were developed under a grant from the US Department of Education, #H323A090005.

However, these contents do not necessarily represent the policy

of the US Department of Education, and you should not assume endorsement by the Federal Government.

Project Officer, Jennifer Coffey, PhD.