Harvesting Data for Problem Solving

REBECCA PIERMATTEIWayne HickmanChristina Jordan

PBIS Maryland Coaches MeetingApril 30, 2014 & May 5, 2014

WHAT CAN WE LEARN FROM DATA?

What we are doing well and what challenges we face

Which students, teachers, systems need more support

Whether our support is successful

In a NutshellKnow what data you need and how to access the

data.

Make sure your data is reliable

Review and share data on a regular basis.

Keep it simple and reasonable.

Use your data to guide your decision making.

Use your data to evaluate individual progress as well as program and school-wide success.

BEFORE YOU START…Resource Mapping

Resource Mapping-Inventory of Current Practices

• What are the practices in place at each tier of the triangle?

• Are they evidence-based practices?• How are you measuring effectiveness of

practices (data)? • Who are the service delivery

teams/personnel (e.g., graduation coach, PALS teacher, Math Coach)

Tier 3

Tier 2

Tier 1

Triangle Activity:

Applying the Three-Tiered

Logic to Your School

7

Practices, Initiatives, Programs for a FEW

Practices, Initiatives, Programs for SOME

Practices, Initiatives, Programs for ALL

BEFORE YOU START…Resource Mapping

Early Warning Indicators

Early Warning IndicatorsCourse

Performance inCore Subjects GPA Credits

FCAT/Concordance

ScoresAttendance

Office Discipline Referrals

AdditionalFactors

On-Track Indicators

On-Track

Meeting all graduation requirements Cs or better in all areas

2.5 or more Meeting credit graduation requirement for grad plan year

Level 3 or Above or concordant scores within the same school year

4% or less absences per quarter or semester

3 or less Level I and/or minor referrals

DisengagementNo extra curricular involvementSubstance AbuseHigh MobilityMental health issuesFree/Reduced lunchFoster/group homeTransient/HomelessParent unemployment Student employmentChanges in behavior/ appearance More recent traumatic eventMissed guidance appointmentsNo show for yearbook picture

At-Risk forOff Track

Lacking 1 graduation requirement

2.0 to 2.49 Behind 1 Credits

Level 2 on FCAT 5% or more absences per quarter or semester

4 or less Level I and/or minor referralsLevel II ODRs per semester

Off-Track

Lacking 2 graduation requirementsFailing 1-3 classes

Less than 2.0 Behind 3 credits Not passed both sections of 10th grade FCAT or retakesNo concordant scores

10% absences per quarter or semester

5 or more Level I and/or Level II ODRs per semester

Highly Off-Track

Lacking 2 or more graduation requirementsCurrently failing 3 or more classes

Less than or equal to 1.5

Behind 4 or more credits

Not passed 10th grade FCAT or retakesNo concordant scores

15% or more absences per quarter or semester

5 or more Level II ODRs for fighting/ profanity/ disruption per semester

ExtremelyOff-Track

Meeting no graduation requirements2-3 Years Behind

Less than or equal to 1.0

Not meeting cohort graduation plan

Not passed 10th grade FCAT or retakesNo concordant scores

20% or more absences per quarter or semester

Established pattern of severe behavior Level II & III ODRs

Part A: SCHOOL-WIDE DATAIdentify what academic and behavioral data you need

ACADEMIC: Homework completion, GPA, Credit Accrual, Benchmark Assessments

BEHAVIORAL: ODRs (Big 5), attendance, suspension/expulsion, minor incidents, nursing and counseling logs

Identify the sources of that data

Identify the person responsible for getting and presenting the data

Identify how often and it what manner it will be shared with the team, faculty, and administration

Part B: WHAT DO WE DO WITH THE DATA?

Celebrate success!

Identify problems with PRECISION using the 5 W’s:What, Where, When, Who, Why

Determine whether your data indicates a need for school-wide practice or small group/individual response?

Develop Solution Options

Create Problem Solving Action Plan

Evaluate Solution

HELPFUL TOOLSData Decision RulesProblem Solving Action Plan

IF: Focus On:

> 35% of students receive 1 or more referralsAverage referrals per student > 2.5

Schoolwide Systems

> 35% of referrals come from non-classroom settings> 15% of students who receive a referral are referred from non-classroom settings

Non-Classroom Systems

>50% of referrals are from classroom settings>40% of referrals come from less than 10% of the classrooms

Classroom Systems

At least 10-15 students have 5 or more referrals Targeted Group Interventions

<10 students with 10 or more office referrals<10 students continue w/referrals after targeted interventionSmall number of students destabilizing overall functioning

Individual Student Systems

EXAMPLEPRECISION PROBLEM STATEMENT:

Many students from all grade levels are engaging in disruption, inappropriate language, and harassment in the cafeteria and hallway during lunch, and the behavior is maintained by peer attention.

Identify 5 W’sIdentify which system you will target

TIER 2/3: Sorting Students into Interventions

ACTIVITY

BEFORE YOU START…Resource Mapping

Early Warning Indicators

Decision Rules for Access to Interventions

Decision Rules for Access to Advanced Tiers

Specific to each intervention

Identify objective variables/criteria as well as the minimum/maximum for eachEX: GPA between 1.0 and 1.5

You may need to scale up or down depending on your capacity

Part C: Sorting Students into Interventions

1. How will students be identified for this intervention? What are the data decision rules for access – criteria used, min and max for each variable?

2. What data do we need?

3. Where will we get the data?

4. Who will be responsible for collecting the data?

HELPFUL TOOLSEARLY WARNING INDICATORS

RESOURCE MAPPING

DECISION RULES FOR ACCESS TO ADVANCED TIERS

Part D: Progress Monitoring and Program Evaluation

1. How will student progress be measured/monitored? Who is responsible for measuring/monitoring progress?

2. What indicators/benchmarks will show that a student is responding to the intervention? Not responding?

3. What indicators/benchmarks will show that a student is ready to exit the program?

4. How will overall program success be measured/monitored?

HELPFUL TOOL

Intervention Tracking Tool

*Illinois PBIS Network

OH NO!What do you do if your data shows

your program is not working?

FIDELITY MEASURESDetermine whether you are

implementing with fidelity(Are you doing it the way you are supposed to be doing it?)

Consider BOQ, SET, PBIS-TIC

Consider intervention specific fidelity measures

IMPORTANT CONSIDERATION

DATA MUST BE RELIABLEEXAMPLE: Behavior Referrals

Define your behaviors

Agreement re: major/minor

Standardized procedures for gathering referrals

All staff trained in how to complete and submit referrals

In a NutshellKnow what data you need and how to access the

data.

Make sure the data is reliable

Review and share data on a regular basis.

Keep it simple and reasonable.

Use your data to guide your decision making.

Use your data to evaluate individual progress as well as program and school-wide success.

Acknowledgements MDS3 is funded by a grant from the USDOE.

Federal Grant CFDA# Q184Y100015

Sheppard Pratt Health System: Rebecca Piermattei, M.S. Rpiermattei@mds3online.org Wayne Hickman, Ed.D. Whickman@mds3online.org Christina Jordan, M.Ed. Cjordan@mds3online.org

Maryland State Department of Education

Johns Hopkins University