Booster/Refresher Training:Evaluation

Benchmarks of Quality Items # 49 – 53

2011-2012

2

Implementation is a Process…

• First 1-2 years (or more), teams work to establish their core, refine their procedures, & develop system-level supports for implementation• Lack of buy-in can slow implementation down

• Critical elements, student outcomes, & integrity of Tier 1 interventions must be monitored at the same time

• Your Action Plan provides accountability– Revisit it, Revise it, Add to it, Evaluate your progress– Items should align to evaluation data

3

Evaluating Tier 1 ImplementationDiagnost

ic Progress Monitoring Outcome

BaselineBoQ

BoQ

PIC

Surveys:•Faculty•Student•Parent

•PBS Walkthrough

•Participation in Reward Activities

•Artifacts of Lessons

•Focus Groups

•Observations

ODRs

Minors

OSS/ISS

Attendance

SESIR

% Students w/ Referrals

Surveys:•Climate•Bullying •Sub.Abuse

Academics

4

Evaluating Systems Change

How well did your leadership team: Teach the discipline system to all staff? Train staff and students on HOW to teach the

expectations/rules/rewards? Provide booster/refresher trainings to staff and

students throughout the school year? Plan a yearly reward/recognition schedule of events? Orient new staff and students? Involve families and community members?

Look at your BoQ Data

Survey faculty: get input &

support

6

Data Review

• How did your team score on Items 49-53 on the BoQ?

• What is working?

• What needs to be improved?

7

Problem-Solving Process

Step 1: Problem Identification

Step 2: Problem Analysis

Step 3: Intervention Design

Step 4: Response to Intervention

Why is it occurring?

What’s the problem?

What are we going to do about it?

Is it working?

8

Step 1: Problem ID

What’s the difference (“gap”) between what you want and what you have

What systems are problematic?The “Big 5”

Referrals by location, problem behavior, time of day, staff & student

Focus on school-wide issuesTarget non-classroom locations at first…Classroom

implementation will be addressed over time

USE YOUR DATAto identify and analyze the problems

9

Step 2: Problem Analysis

Brainstorm a variety of reasons that may contribute to the problem:Instruction

Curriculum

Environment

Learner

Choose the reason with data to support it, aligns with the proposed function of the behavior, and can be addressed with current resources.

10

Instruction Curriculum Environment Learner

Relevancy, Content, Difficulty, Ease

Opportunity to Respond,Location,Schedule of Lessons,Review of Skills,Feedback on Skills,Pre-Corrections/Cues

Reward System,Consequence System,Visual Cues,Physical Environment,Staff Proximity/Line of SightCommunity Characteristics

Reinforcement/ Punishment/ Instructional History;Preference for rewards

11

DOMAINS Hypothesis RReview

IInterview

OObserve

TTest

IInstruction

CCurriculum

EEnvironment

LLearner

A variety of data sources should be used to validate a

variety of hypotheses

Validating Hypotheses: ICEL by RIOT

12

Step 3: Intervention Design

Once problem is identified, determine why it’s occurring:Proactive: prevent the behaviors from recurring; look at

the antecedents and environment

Educative (Replacement Behaviors): teach/re-teach desired/replacement behavior

Reinforcement (Encourage appropriate behaviors and discourage problem behaviors): only reinforce desired behaviors, address function of behavior

How will you know when you’re successful?

13

Step 4: Response to Intervention

How will you know if your school intervened with fidelity?

How will you know when you’re successful?Did we meet the goal?Do we need to develop a new plan?How accurate were our problem ID & hypotheses?

Were we accurate in identifying the function?Did our interventions address the function of the

inappropriate behavior?Or, develop a plan to fade out the intervention if it

was successful

14

https://demo.flrtib.org

15

Action Plan!Refine Evaluation Plan

Record action items for low-scoring Benchmarks items (49-53)

Outline your plan how you will use data for decision making across the school year

Record action items to inform/involve your stakeholders (faculty, students, families) Implementation Tips may help

BoQ Element: Evaluation