impacting outcomes for students with the most challenging behaviors through schoolwide pbs: osep’s...
TRANSCRIPT
Impacting Outcomes for Students with the Most Challenging Behaviors through Schoolwide PBS: OSEP’s Model Demonstration Projects
Lucille Eber, Statewide Director Illinois Statewide Technical Assistance Center PBIS NetworkJennifer Doolittle, Project OfficerOffice of Special Education Programs
Timeline of OSEP’s PBS Investments 1997: IDEA reauthorized 1998: OSEP funds the Center on Positive Behavioral Interventions and Supports
(PBIS). Provides a framework for schoolwide behavior support practices
2002: OSEP funds the Center for Evidence-based Practice: Young Children with Challenging Behavior
2004: “PBS Blueprint for Practice” emphasizes developing State and district level infrastructures that can promote large-scale implementation and sustainability
2006: Tertiary Behavior Model Demonstration Sites 2008: Technical Assistance Center for Social-Emotional Intervention for Young
Children (TACSEI) Scaling up evidence-based behavior support practices at the preschool level IDEA performance measures for social-emotional outcome for birth to five
The Future Continue to scale-up school-wide PBS Assist schools to identify and implement more tertiary-level, intensive behavior support
strategies.
OSEP’s Model Demonstration Projects (MDPs) Model Demonstration Coordinating Center
Creating cross-site evaluation plans, instruments, and procedures to assess the context, implementation, and efficacy of each model and across models within a cohort.
Identifying key issues in translating research to practice and analyzing and synthesizing data across MDPs and cohorts
Applying multiple analytic methods to answer evaluation questions regarding both the efficacy of models and bridging the gap between research and practice.
Producing high-quality, useful, and accessible products that communicate findings to key audiences.
Cohorts Each year a new cohort of three to four MDPs is added; each cohort has a
different focus: 2006: 1st cohort is focusing on progress monitoring for preschool through
4th grade in general and special education classrooms. 2007: 2nd cohort is developing tertiary behavior interventions for
elementary/middle schools for students with challenging behaviors. 2008: 3rd cohort will develop, implement and evaluate early childhood
language interventions. Documenting model development, model implementation, and model
outcomes for each cohort, and analyzing MDP experiences and results across cohorts Help OSEP bridge the gap between identifying evidence-based practices and
achieving their widespread use.
Tertiary Behavior MDPs Illinois PBIS Network and the University of
Kansas The University of Oregon The University of Washington
Tertiary Behavior MDPs Big Ideas School-wide intensive PBS approach guided by a three-tiered
prevention model Individualized, function-based behavior support to children who
exhibit the most challenging behaviors and have not been responsive to primary or secondary prevention efforts
Emphasis on collecting and using data for decision-making Systematic strategies for professional development TA that improves student behavior to help schools promote student
learning and other positive outcomes Cost effective and efficient process for school districts to implement
and sustain A response to intervention (RtI) logic model to show student
progress
RTI for Behavior and Academics Key components
Evidence-based instruction and supports for all students Universal screening to determine which students are not
meeting benchmarks Targeted instruction and support that goes beyond what
all students receive Progress monitoring Intensive and individualized instruction and support for
students who are still not making progress - special education?
Decision points throughout
The Kansas-Illinois
SW-PBS Tertiary Demonstration
Center:
A Response to Intervention (RtI)
Continuum of Support Model
CEC Boston April 4, 2008
Lucille Eber, Illinois PBIS Network
Wayne Sailor, University of Kansas
K-I Center Team Leaders
• Jamie Bezdek, University of Kansas• Kimberli Breen, IL PBIS Network• Jen Rose, Loyola University-IL PBIS Network• Amy McCart, University of Kansas
Evaluation:
• Kelly Hyde (SIMEO)• Holly Lewandowski (PoI and SWIS data)
Big Ideas for this Session
1. How the K-I Center is applying the RtI approach to both behavior and academics to ensure tertiary capacity
2. Implementation experiences and data from IL (Year One and Year Two)
3. What the K-I Center hope to “deliver” in terms of knowledge, tools etc.
Does building a school-wide system of PBIS increase school’s abilities to effectively educate students with more complex needs?
What systems, data and practice structures are needed to ensure that positive behavior support being applied in needed dosage for ALL students?
Key Questions
80-90% 80-90%
Tertiary Interventions•Individual Students•Assessment-based•High Intensity
Tertiary Interventions•Individual Students•Assessment-based•Intense, durable procedures
Secondary Interventions•Some students (at-risk)•High efficiency•Rapid response•Small Group Interventions• Some Individualizing
Secondary Interventions•Some students (at-risk)•High efficiency•Rapid response• Small Group Interventions• Some Individualizing
Universal Interventions•All students•Preventive, proactive
Universal Interventions•All settings, all students•Preventive, proactive
1-5% 1-5%
5-10% 5-10%
Academic Systems Behavioral Systems
School-Wide Systems for Student Success
A Response to Intervention Model
Positive Behavior Interventions & SupportsA Response to Intervention (RtI) Model
Universal School-Wide Assessment
School-Wide Prevention Systems
Secondary
Tertiary
SWIS & other
School-wide data
BEP & group
Intervention data
SIMEO tools: HSC-T, RD-T
Small group interventions (CICO, SSI, etc)
In
terv
entio
nAssessment
Revised March 2008 IL-PBISAdapted from T. Scott, 2004
Functional assessment tools/Observations/scatter plots etc.
Group interventions withindividualized focus (CnC, etc)
Simple individual interventions(Simple FBA/BIP, schedule/curriculum changes, etc)
Multiple-domain FBA/BIP
Wraparound
3.5.08
Core Features of a Response to Intervention
(RtI) Approach• Investment in prevention• Universal Screening• Early intervention for students not at “benchmark”• Multi-tiered, prevention-based intervention approach• Progress monitoring• Use of problem-solving process at all 3-tiers• Active use of data for decision-making at all 3-tiers• Research-based practices expected at all 3-tiers• Individualized interventions commensurate with
assessed level of need
Continuum of Support for Secondary-Tertiary Level
Systems1. Group interventions (BEP, social or academic skills
groups, tutor/homework clubs, etc)2. Group Intervention with a unique feature for an
individual student, (BEP individualized into a Check & Connect; mentoring/tutoring, etc.)
3. Simple Individualized Function Based Behavior Support Plan for a student focused on one specific behavior (simple FBA/BIP-one behavior; curriculum adjustment; schedule or other environmental adjustments, etc)
4. Complex Function-based Behavior Support Plan across settings (i.e.: FBA/BIP home and school and/or community)
5. Wraparound: More complex and comprehensive plan that address multiple life domain issues across home, school and community (i.e. basic needs, MH treatment, as well as behavior/academic interventions) multiple behaviors
3.8.08
ILLINOS SW-PBS History
• The Context for Implementing the Tertiary Demo process……
IL PBIS Schools Over Nine Years:
Trained & Partially or Fully Implementing
23
120184
303
394444
520587
654
0
100
200
300
400
500
600
700
Year 1 9/98
Year 2 9/00
Year 3 9/01
Year 4 6/02
Year 5 6/03
Year 6 6/04
Year 7 6/05
Year 8 6/06
Year 9 6/07
num
be
r o
f sc
ho
ols
Implementing Tertiary Demos: The IL Context for
IL PBIS Expansion History
June 30, 2005
444 schools in 143 districts
92 new schools trained
520 schools in 155 Districts 97 new schools trained in FY06 12 new districts
654 schools in 170 72 schools trained in FY07 15 new districts
744 Schools in 196 Districts Approximately 90 schools trained in 1st half
of FY08 Approximately 26 new districts in 1st half of
FY08
June 30, 2006
June 30, 2007
January, 2008
Illinois PBIS Schools
74%
14%
12%
83%
12%
5%
50%
60%
70%
80%
90%
100%
% o
f stu
de
nts
with
OD
Rs
Partia lly Implementing
(n=58)
Fully Implementing (n=141)
0-1 ODRs 2-5 ODRs 6+ ODRs
Mean Percentage of Students with Major ODRs 2006-07, Statewide
The differences between fully and partially implementing schools were statistically significant in all three levels of ODRs
(0-1 ODR, Mann-Whitney U=3035.0, p=0.004; 2-5 ODR, Mann-Whitney U=3050.0, p=0.005; 6+ODR, Mann-Whitney U=3062.0, p=0.005).
149170
187151
195
0
50
100
150
200
250
2002-2003 2003-2004 2004-2005 2005-2006 2006-2007
Years
Num
ber
of s
cho
ols
Schools Completing Profiles
# of Schools Reporting Small Group and Indiv idual Interventions
Illinois PBIS Schools
Illinois PBIS Schools Completing School Profile Forms &
Implementing Secondary/Tertiary Interventions across Five
Years
Comparison of Partial & Fully Implementing
Schools
on Suspensions/Expulsions FY07 per 100 Students11.2
5.7
0
2
4
6
8
10
12
Partial (n=58) Full (n=140)
Implementation
Num
be
r of s
usp
ens
ions
pe
r 100
st
ude
nts
Illinois PBIS Schools
Comparing School Safety Survey
Partial vs. Full Implementation
4641
73 78
0
20
40
60
80
100
Partial (n=20) Full 80/80 (n=62)
Partial vs Full Implementation
Perc
enta
ge o
f Risk
and
p
rote
ctio
n fa
cto
r
Risk factor Protection factor
Illinois PBIS Schools
Illinois PBIS Schools
59.3
68.4
54
56
58
60
62
64
66
68
70
Partial (n = 89
schools)
Full (n = 120
schools)
Level of PBIS Implementation
Pe
rce
nta
ge
of st
ud
ents
The difference between the two types of schools was significant (t=3.72, df=159, p<0.001).
Illinois 2005-06 Proportion of Students who Meet
or Exceed Third Grade ISAT Reading Standard
Illinois PBIS Schools
65.6
86.6
0
10
20
30
40
50
60
70
80
90
100
Partial (n = 160
schools)
Full (n = 157 schools)
Level of PBIS implementation
Perc
en
tag
e o
f sc
ho
ols
tha
t m
et
AYP Findings
suggest that fully implementing PBIS schools met AYP at a significantly higher percentage than partially implementing schools (χ2=19.17, df=1,
p<.001).
Illinois 2005-06 Proportion of Schools
that Met AYP
Small Group & Individual Interventions Rated "High" or
"Very High" in Fully & Partially Implementing PBIS
Schools 2006-07142
42
0
50
100
150
Fully Implementing
Schools (n= 70 schools)
Partia lly Implementing
Schools (n= 24 schools)
Level of implementation
Nu
mb
er o
f in
terv
en
tion
Illinois PBIS Schools
Small Group Interventions Rated as "Very High” &
"High“ in
Fully & Partially Implementing PBIS Schools 2006-07 73
21
010203040
50607080
Fully implementing (n = 47
schools)
Partially implementing (n = 14
schools)
Level of implementation
Num
ber o
f inte
rve
ntio
ns
Illinois PBIS Schools
Individual Interventions Rated “Very High” & “High”
in Fully & Partially Implementing PBIS Schools 2006-07
69
21
010
2030
40506070
80
Fully implementing (n = 44
schools)
Partially implementing (n = 14
schools)
Level of implementation
Nu
mb
er o
f in
terv
en
tion
s
Illinois PBIS Schools
SIMEO Database (Systematic Information Management of Education Outcomes)
Technical Features: Database Development
online data collection and graphing database system for individual student receiving intensive level planning and supports
Challenges….
69.47%
20.00%
10.53%
9.82%
36.61%
53.57%
0%
20%
40%
60%
80%
100%
% Students In Group % ODRs From Group
0 or 1 2 to 5 6+
Mean Percentage of Students for Tertiary Demo School A
by Major ODRs 2005-06 Grades K-6 (285 students)
Tertiary Demos
76.79%
17.75%
5.46%
15.36%
46.38%
38.26%
0%
20%
40%
60%
80%
100%
% Students In Group % ODRs From Group
0 or 1 2 to 5 6+
Mean Percentage of Students for Tertiary Demo School A
by Major ODRs 2006-07 Grades K-6 (293 students)
Tertiary Demos
Examples of Ineffective or
Weak
Secondary/Tertiary Systems• Referrals to Sp.Ed. seen as the
“intervention” beyond universals
• FBA seen as required “paperwork” vs. a critical step in designing effective interventions
Some “Big Picture” Challenges?
• Low intensity, low fidelity interventions for behavior/emotional needs
• Habitual use of restrictive settings (and poor outcomes) for youth with disabilities
• High rate of undiagnosed MH problems (stigma, lack of knowledge, etc)
• Changing the routines of ineffective practices (systems) that are “familiar” to systems
Investing in Tertiary Demos?
• ISBE is investing• ICMHP is investing• USDOE-OSEP is investing
We need replicable Systems, Data, and Practices so we can effect long term
change.
Tertiary Demos
Secondary Training Events A Two Year Comparison
0
200
400
600
800
2005-06 2006-07
# p
artic
ipa
nts
0
20
40
60
Tra
inin
gs
Secondary Participants
Secondary Trainings
Tertiary Training EventsA Two Year Comparison
0
200
400
600
800
1000
2005-06 2006-07
# p
artic
ipa
nts
0
10
20
30
Tra
inin
gs
Tertiary Participants
Tertiary Trainings
A Focus on Tertiary Impacts Implementation at All Levels
• Notable progress was observed in tertiary demo schools’ implementation of PBIS.
• Building-based teams met frequently to action plan and significant gains were made during year one.
• The Illinois PBIS Phases of Implementation Tool is
being used by schools to self-assess their systems, data and practices and guide their implementation.
• As schools invest in developing tertiary structures,
they are also taking steps to improve their universal and secondary systems.
Tertiary Demo School Reduces ODRs & Increases Simple Secondary Interventions
36
551
0
5
10
15
20
25
30
35
40
Aug to Nov 2006 Aug to Nov 2007
num
be
r o
f st
ud
ents
2-5 ODRs 6+ ODRs CICO*
*CICO = Check in, Check Out
IL Tertiary Demo
Results of Implementation of Wraparound within SW-PBS in
IL• Three year pilot• Enhance SOC wraparound approach
– data-based decision-making as part of wraparound intervention
• Development of strength-needs data tools
• Web-based system
Wraparound
IL PBIS Tertiary Demos-07
Tertiary Interventions Linked to Immediate
& Sustainable ODR Decreases
2
125
50
0
25
50
75
100
125
150
Time1 (N=26) Time 2 (N=26) Time 3 (N=10)
Cum
ula
tive
Inc
ide
nc
e
Office Discipline Referrals
Immediate & Sustainable Change Noted in Placement
Risk
1.3
1.78
1.5
1
1.25
1.5
1.75
2
Baseline Time2 Time 3
High Risk
Low/No Risk
(n = 19)
Wraparound-07
0.5
2.37
3.87
2.84
0.79
1.38
0
1
2
3
4
Baseline (n=19) Time 2 (n=19) Time 3 (n=8)
ODRs OSSs
Avg #
of
ep
isodes
Wraparound-07
School Risk Behaviors Substantially Decline
for Student Engaged in Wrap
IL Wraparound Data-07
Positive Classroom Behavior & Academic Achievement Linked
2.8
2.212.45
2.83
2.192.13
1
2
3
4
Baseline (n=26) Time 2 (n=26) Time 3 (n=12)
Classroom Behav ior Functioning Academic Achievement
Always
Never
0
5
10
15
20
25
30
FY 05 (N=18) FY 06 (N=26) FY 07 (N=26)
PBIS Network Staff School Social Worker PBIS Coach Other School Personnel
Shift in Responsibility for Individual Student
Data Management at Tertiary Demo Sites
IL PBIS Tertiary Demos
“Andy”
Using Data to Keep the Team Moving
“Celebrate Success of current plan”
“Andy”
Next Steps
• Team will use data to plan for academic supports.
• Lower intensity of some supports
• Transition from frequent adult feedback to self-monitoring to boost self-confidence and feel/be less dependent on adults.
“Mary Ellen”
Home, School, Community Tool
SIMEO -Educational Information Tool
00.5
11.5
22.5
33.5
Baseline Time 2 Time 3 Time 4
1=N
eve
r 4
= A
lwa
ys
attends school
participates in extracurricular activities
appropriate behavior in unsupervised settings
“AJ”
• 0 ODRs (from 3)• 0 Time-outs (from 22)• Passing grades in all classes (from D’s & F’s)• Parents report that: “this is different”• Improved partnerships with community
service providers• MH partner participating and providing
effective strategies
“Henry”Reason Referred to Tertiary
Supports “Henry”, an elementary school student,
had: extremely poor attendance failing grades poor homework completion trouble with the law in the community and had a court assigned probation officer
and a mandated Department of Children and Family Services (DCFS) counselor
“Henry”
Engagement and Team Preparation
• Wraparound team initially included – Henry– his mother – the school social worker – his primary classroom teacher – school principal, bi-lingual liaison– district SWPBS tertiary-tier coach
• Team met consistently to identify strengths, big needs and develop a wraparound plan for Henry.
• Henry’s strengths identified by team included:– a good relationship with his teacher – responsiveness to positive attention from adults he
likes – leadership among his peers – and effective self-advocacy
• Henry’s “big needs” as identified by the wraparound team: – Henry needs to feel as if he fits in with the other kids
at school– Henry needs to feel successful at school– Henry to be invested in his education
“Henry”
Identifying Strengths and Big Needs
• Henry was included in the “Check-and-Connect” intervention which was being delivered to other students in the school
• In addition, Henry and his teacher talked about individual behavior goals listed on his daily point card.
• He was put on “safety patrol” in which he was an older youth selected to be a positive role model to help monitor and improve his behavior in the hallways.
• Because Henry’s voice was important, the suggestion that he work with younger students in their classrooms was set aside due to Henry’s lack of interest.
“Henry”
Ongoing Plan Implementation and Refinement
• Henry’s progress was monitored through:– Office discipline reports– Attendance– Grades– DIBEL scores – Check In Check Out behavior card points– SIMEO Tools
“Henry”
Progress Monitoring
• From second quarter to third quarter, with wraparound in progress, Henry’s grades and attendance began to increase – Spelling: 15% to 40% – Math: 15% to 48.5% – Reading: 20% to 63%– His DIBELS score increased from 55 words per
minute in the fall to 67 words per minute in the winter.
– Attendance:15% in 1st quarter 60% in 2nd quarter, 75% in 3rd quarter
“Henry”
Initial Outcomes
Henry’s Risk of Placement Data
Referral-Disposition Tool (SIMEO)
Henry’s Improved Behavior and Emotional Functioning at Home
Henry’s Improved Behavior and Emotional Functioning at School
Henry’s Improved Behavior and Emotional Functioning In the
Community
Educational Environment Data (EE)
• A key item in IL State Performance Plan for feds
• More districts to be “flagged” for monitoring
• Tertiary demo activities focus on IL SPP data points
0
50
100
150
1999-00 2000-01 2001-02 2002-03 2003-04 2004-05
Num
be
r of S
tud
ents
Monitor Resource Self contained
Six Year Comparison of Least Restrictive Environment
Sparta School District
Changes in Least Restrictive Environment
Dewey Elementary School
27
45
16
5
78%60%
0
5
10
15
20
25
30
35
40
45
50
2003-04 2004-05
num
be
r stu
de
nts
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
ISA
T sc
ore
students in SPED < 21% of day students in SPED 21-60% of day ISAT scores
• First step is accessing the data
• Next is discussing with range of stakeholders and determining accuracy or how to make it accurate
• Possible tools/procedures to make a difference
EE Data(continued)
Similar to how we got started with ODR data
• clean up data (e.g. ODR form)
• review data trends and ask questions;
Getting Started with Data-Based Decision-Making
with EE Data
Getting Started (continued)
• decide what it means by those who “live” the data;
• decide what data points to focus on
• design actions that seem likely to effect change;
• monitor/revise action plan
Why We Need MH Partnerships
• One in 5 youth have a MH “condition”• About 70% of those get no treatment• School is “defacto” MH provider• JJ system is next level of system default• 1-2% identified by schools as EBD• Those identified have poor outcomes• Suicide is 4th leading cause of death among
young adults
Why Do We Need to Go
Beyond Use of ODRs?
• Use of “alternative” discipline responses; often w/o documentation
• Over use of “Special Education” placement w/o adequate dosage of interventions
Why Do We Need to Go
Beyond Use of ODRs? (continued)
• High rate of unidentified MH problems
• Youth get identified only after “crisis” which makes it harder and more “costly” to intervene.
The Systematic Screening for Behavior Disorders (SSBD)
(Walker and Severson, 1992)
• Developed as a school-wide (Universal) screening tool for children in grades 1-6
– Similar to annual vision/hearing screenings
Background
• Identifies behaviors that may impede academic and social functioning
• Leads to earlier intervention
• May reduce need for formalized, lengthy “requests for assistance” by using data to identify youth
Implementation
• Between early September-first of November, completed screenings in 6 districts and 18 schools
• Initial results indicate that approximately 5%-10% of students enrolled in grades 1-6 were identified by the SSBD
• A Middle school case example:
– Approximately 320 students enrolled in sixth grade were screened using the SSBD
– 38 six graders or 11% passed gate two
Implementation
• Currently, school-based secondary teams are using SSBD data to implement low-intensity interventions (e.g., check-in/check-out)
Implementation
Ensuring Capacity at All 3 Tiers
• Begin assessment and development of secondary and tertiary tiers at start-up of universal– Assess resources and current practices
(specialized services)– Review current outcomes of students with higher
level needs– Position personnel to guide changes in practice– Begin planning and training with select personnel
• All 3 tiers addressed at all district meetings and at every training
Requirements for IL Tertiary Demos
• District Commitment• Designated Buildings/District Staff• External Tertiary Coach/Coordinator• Continuum of Skill Sets (training, guided
learning, practice, coaching, consultation)• Commitment to use of Data System
– Going beyond ODR’s (i.e. SSBD)– Self assessment/fidelity– SIMEO-Student Outcomes
District-wide Secondary/Tertiary
Implementation Process• District meeting quarterly
– District outcomes– Capacity/sustainability– Other schools/staff
• Building meeting monthly– Check on all levels– Cross-planning with all levels– Effectiveness of practices (CICO/BIP/Wrap, etc)
• Secondary/Tertiary Coaching Capacity• Wraparound Facilitators
System Data to Consider
• LRE– Building and District Level– By disability group
• Other “places” kids are “parked”– Alternative settings– Rooms w/in the building kids are sent
• Sub-aggregate groups– Sp. Ed.– Ethnicity
Ongoing Self–Assessment of Secondary/Tertiary Implementation
Building Level:• IL Phases of Implementation (PoI) Tool • IL Secondary/Tertiary Intervention Tracking Tool• Sp. Ed Referral Data• Suspensions/Expulsions/Placements (ongoing)• Aggregate Individual Student Data (IL SIMEO data)• LRE Data trends• Subgroup data (academic, discipline, Sp. Ed. Referral, LRE, etc)
District Level:• Referral to Sp.Ed. Data• LRE Data (aggregate and by building)• IL Out-of-Home-School-Tracking Tool (multiple sorts)• Aggregate SIMEO data• Aggregate PoI Data