2014-2017 School Climate Transformation Grant Implementation Survey

Summary

Prepared by Jen Freeman, Rob Horner, & George Sugai

OSEP TA Center on PBIS

March 2017

Background

In October 2014, the office of Safe and Healthy Students funded 12 states and 71

districts to implement a Multi-tiered Behavior Support Framework. The National PBIS

Technical Assistance (TA) Center has received funding to provide technical assistance to

states and districts across sites.

To assess the ongoing technical assistance needs and implementation status across

School Climate Transformation Grant (SCTG) recipients, the National PBIS Technical

Assistance (TA) Center conducted implementation surveys three times per year (Nov,

March, May) during the first year of the project, two times per year (Nov, March) during

the second year and once per year (March) for the duration of the project. All SEA and

LEAs were sent a link to an online survey and asked to complete this information

electronically. The purpose of this report is to summarize these data from October 1,

2014 through March 30, 2017 and describe how the TA center is using this information

to guide the ongoing organization of TA supports across sites.

As of March 2017, School Climate Transformation Grants are supporting work in

at least 151 districts and 1418 schools.

Elementary

School

Middle

Schools

High

Schools

Other Districts

LEA Grants 585 234 154 81 70

SEA Grants 157 109 93 7 81

Total 742 341 247 88 151

Response Rate

Thank you to all SEA and LEA teams who submitted survey data. The TA center

will continue to collect implementation survey data annually for the duration of the

project, and we encourage SCTG recipients to continue to discuss TA needs with their

TA providers as they arise.

TA

Worksheet Nov 2014

Survey

March 2015

Survey

May 2015

Survey

Nov 2015

Survey

March 2016

Survey

March 2017

Survey

SEA 5/12

(42%)

9/12

(75%)

10/12

(83%)

8/12

(67%)

9/12

(75%)

11/12

(92%)

8/12

(67%)

LEA 19/71

(27%)

40/71

(56%)

49/71

(69%)

54/7

(76%)

37/71

(52%)

42/70

(60%)

57/70

(81%)

67%

92%75%

67%83%75%

42%

81%

60%51%

76%69%56%

27%

0%20%40%60%80%

100%

Response Rate

SEA

LEA

Results In the following charts, we compare the results of implementation surveys

from Nov 2014- March 2017 in the areas of implementation steps taken and evaluation plan components. Additionally, we provide a summary of practices included in SEA and LEA implementation plans.

0% 20% 40% 60% 80% 100%

Established Leadership Team

Established Implementation Team

Identified Coordinator

Established Support of Key Stakeholders

Selected Specific Districts/Schools forParticipation

Established a Plan (e.g. monthly leadershipteam meetings, training schedule)

Completed SCA

Agreed Upon State Evaluation Plan

Districts established leadership teams

Established Coaching Capacity forParticipating Schools

One leadership team with budget authorityacross multiple funding streams (if…

Plan for assessing district and schoolreadiness for tier 2 systems implementation

Plan for assessing district and schoolreadiness for tier 3 systems implementation

District support for tier 2 and tier 3 systemsimplementation in participating schools

Establishing coaching capacity and supportfor tier 2 and tier 3 systems implementation?

State data system capacity to support tier 2and tier 3 systems implementation?

Agreed upon process for aligning andintegrating multiple initiatives to avoid…

State leadership team includerepresentatives from multiple departments…

SEA: Percent of Sites Reporting Implementation Steps in Place

Nov 2014 (N=9)

March 2015 (N=10)

May 2015 (N=8)

Nov 2015 (N=9)

March 2016 (N=11)

March 2017 (N=8)

0% 20% 40% 60% 80% 100%

Established Leadership Team

Established Implementation Team

Established Support of Key Stakeholders

Selected Specific Districts/Schools forParticipation

Established a Plan (e.g. monthly leadershipteam meetings, training schedule)

Completed DCA

Agreed Upon District Evaluation Plan

Plan to Complete TFI in each School

ODR systems provides disaggregation byabuse of substance/weapons

Established Coaching Capacity forParticipating Schools

Receiving other Federal Funding

Leadership team with budget authorityacross multiple federal funding streams

Plan for assessing school readiness for tier 2systems implementation

Leadership teams established for tier 2implementation in participating schools

Plan for assessing school readiness for tier 3systems implementation

Leadership teams established for tier 3implementation in participating schools

Plan for establishing coaching capacity andsupport for advanced tier systems…

Data system capacity to support tier 2systems implementation

Data system capacity to support tier 3systems implementation

Identified evidence-based tier 2 and 3practices

LEA process for aligning and integratingmultiple initiatives

School based process for aligning andintegrating multiple initiatives/practices

LEA: Percent of Sites Reporting Implementation Steps in Place

Nov 2014 (N=40)

March 2015 (N=49)

May 2015 (N=54)

Nov 2015 (N=37)

March 2016 (N=42)

March 2017 (N=57)

0% 20% 40% 60% 80% 100%

School Climate

Fidelity of Implementation

Student Behavior Outcomes

District Level Capacity for Implementatin

Special Education Referrals

Mental Health Referals

Arrests or Law Enforcement

Disproportionality

Attendance

Other

SEA: Percent of Sites Reporting Evaluation Plan Component

Nov 2014 (N=9)

March 2015 (N=10)

May 2015 (N=8)

Nov 2015 (N=9)

March 2016 (N=11)

March 2017 (N=8)

0% 20% 40% 60% 80% 100%

School Climate

Fidelity of Implementation

Student Behavior Outcomes

Special Education Referrals

Mental Health Referals

Arrests or Law Enforcement

Disproportionality

Attendance

Other

LEA: Percent of Sites Reporting Evaluation Plan Component

Nov 2014 (N=40)

March 2015 (N=49)

May 2015 (N=54)

Nov 2015 (N=37)

March 2016 (N=42)

March 2017 (N=57)

* Other includes: Violence Prevention

• Other includes: Universal Screener, SAP groups (Student Assistance Program),

Prevention Dimensions, QPR, Hope Squads, Mindfulness, Conscious Discipline,

Ripple Effects, TLIM, Suicide Prevention and Character Education, Educational

Wraparound, Responsive Classroom, Diversity & Cultural Competence Training,

Project Green Dot, Life skills, Second Step

0% 20% 40% 60% 80% 100%

Positive Beahvior Interventionsand Supports

Restorative Practices

Trauma Informed Care

Mental Health First Aid

Social Emotional Learning

RENEW

Bully Prevention

Other

SEA: Percent of Sites Reporting Implementation Plan Component

March 2015 (N=10)

May 2015 (N=8)

Nov 2015 (N=9)

March 2016 (N=11)

March 2017 (N=8)

0% 20% 40% 60% 80% 100%

Positive Beahvior Interventionsand Supports

Restorative Practices

Trauma Informed Care

Mental Health First Aid

Social Emotional Learning

RENEW

Bully Prevention

Other

LEA: Percent of Sites Reporting Implementation Plan Component

March 2015 (N=49)

May 2015 (N=54)

Nov 2015 (N=37)

March 2016 (N=42)

March 2017 (N=57)

Technical Assistance Data Summary

What is a TA Event?

TA support offered in person or electronically (web/email/phone) to one or more

LEAs or SEAs which approximates 30 mins or more OR contains significant content

information as determined by the TA provider.

Number of TA Events to Date:

Total number of TA Events 1445

Number of unique LEAs contacted 71/71

Number of unique SEAs contacted 12/12 plus 3 additional states that have

multiple LEA grants but no SEA grant

Who Received TA?

What Type of TA was provided?

394 395

21

392

0

50

100

150

200

250

300

350

400

450

Leadership team Grant Coordinator Evaluator Other

Nu

mb

er o

f T

A E

ven

ts

TA Recipients

166

280

57

165

482

41 22

0

100

200

300

400

500

600

Nu

mb

er o

f T

A E

ven

ts

Type of TA Provided

What was the length of TA events?

What was the primary purpose of TA events provided to date?

What was the secondary purpose of TA events provided to date?

174214

420

75

227

0

100

200

300

400

500

1/2 hr or less 1/2hr-1hr 1-3 hrs Half day Full dayNu

mb

er o

f T

A E

ven

ts

Length of TA Event

58 32 31

238

108

180

248

435

70

050

100150200250300350400450500

Nu

mb

er o

f T

A E

ven

ts

Primary Purpose of TA Events

41

16 14

122 125117 112

54

16

0

20

40

60

80

100

120

140

Nu

mb

er o

f T

A E

ven

ts

Secondary Purpose of TA Events

Webinars

In addition to site based TA the National PBIS TA center has provided 18

webinars and OSHS has provided 1 webinar available to all SCTG recipients. Three

additional webinars are planned for this academic year. In addition, the TA center has

hosted 4 community of practice calls for SEA sites. Materials and videos of all webinars

are available at http://www.pbis.org/sctg/sctg-events. The table below indicates the

number of webinar attendees and how many times each webinar video has been

downloaded. The graphs summarize the webinar ratings by SEA and LEA attendees. The

final table summarizes suggestions for future webinar topics.

Webinar Topic Number of Attendees Number of Video

Views/Material

Downloads

Performance Reporting 128 321/858

Designing Evaluation Plans 105 270/821

Developing State and District

Leadership Teams

80 318/1065

Implementation Action

Planning

85 134/1773

Developing Training and

Coaching Capacity

70 273/1147

Setting up Implementation

Success (9/17)

164/611

Positive Classroom Behavior

Support (10/16)

580/1338

Selecting and Implementing

Evidence Based Practices

(11/19)

276/729

Resources for Enhancing

Equity in School Discipline

(1/21)

448/687

High School Systems (2/18) 343/626

Building District Capacity for

Multi-Tiered Behavioral

Frameworks (3/17)

306/445

Tier II Overview: Readiness,

Data Decisions, & Practices

(4/21)

357/553

Systematic Screening for

Emotional/ Behavioral

Challenges in Tiered Systems

(5/19)

364/872

Systems to Support Teachers'

Implementation of PBIS in the

Classroom (9/22)

31 141/259

Building and Measuring

Regional and District Capacity

to Implement MTSS

16 52/101

Interconnected Systems 30 148/254

Framework

Integrating Trauma Informed

Support in MTSS

63 143/271

Restorative Practices in PBIS 62 57/359

Tier 2 and Tier 3 Support

Systems

43 Not yet posted/119

Equity and MTSS April

Leadership Capacity for MTSS May

Family and Community

Engagement

June

SEA Community of Practice Topic

Attendees Number of Video Views

District Action Planning ECS didn’t facilitate n\a Aligning Multiple initiatives 15 254

High school challenges 12 223

Rural challenges 18 82

3

4

3

2

3

1

2 2

3

1

4

5

3 3

2 2

4

3

4

3

4

3 1

6

5

6

5

4

2

1

1 1

4

3

3

0

1

1 1 1

3

4

4

3

2 2

4 2 2

3

4

1

2

1

3

0

2

4

6

8

10

12SEA Webinar Ratings

Did not attend or access

Not helpful for our project

Somewhat helpful for our project

Extreemly helpful for our project

29

18 18

23 22

1418 17 16

12

19

24

14

29

1512

17 17

15

2119

11

18

14

13

109

8

1913

14

9

6

1912

8

0

01

1

1

0

0

02

1

00

0

1

1

3

0

3

3

6 811

5

119

12 12

18

13 15

24

13

29

17

22 24

0

10

20

30

40

50

60

LEA Webinar Ratings

Did not attend or access

Not helpful for our project

Somewhat helpful for our project

Extreemly helpful for our project

Suggestions for Future Webinars

SEA LEA

• Using ESSA to sustain PBIS

• Implementation strategies for integrating Trauma informed practices into MTBF

• Information on manageable universal screeners

• coaching for implementation

• Sustainability Scaling up Strengthening classroom integration of social-emotional-behavioral needs of students

• Integration of Social-emotional learning within PBIS

• More on Alignment - academic and mental health

• More Tier II Support Strategies

• Culturally Responsive MTBF Reaching parents with MTSS

• More on Interconnected Systems Framework, because I am new to the Project Director role, and have not taken participated in the other webinars.

• More program evaluation - Steve Goodman

• Universal Screeners

• Additional webinars on integrating multiple initiatives

• More webinars on supporting students in the classroom with mental health

• Trauma Sensitive Training; How to deal with implementation roadblocks; Tier 3 systems

• Scheduling at all levels to support tiered instruction

• Expanding Tier 2 Interventions SAIG groups, Academic Seminar, Brief FBA-BIP

• We loved the Trauma Informed Interventions webinar (not listed).

• Building capacity for implementation of function based Tier 2 interventions

• NA

• Webinars specific to aligning the various interventions (RTI, Restorative Practices, Bullying, CICO, SEL, etc.)

• Restorative Practices, Building capacity and sustainability.

• Tier Two and Three Readiness Effective practices for Tier 2 and 3 supports/ interventions

• Sadly, I have not had the luxury of time to watch nor research what webinars there are to research.

• Advanced Tier II/III Interventions

• Alternative funding sources Staff Buy-In

• Data Based Decision Making

• Integrating mental health systems and data collection for Tier 3

• Tier II and Tier III trainings --- What do these tiers look like?

• Additional Tier 2 & 3 Supports

• Additional information about Evidence based Tier 2 and 3 interventions

• Sustainability More on Mental Health Framework

• Continue classroom strategies/management

• I loved the webinar on Trauma Informed Approaches. Our

team is working on creating Trauma Sensitive Schools and giving classroom teachers the tools to help them work with our students who have experienced trauma. Anything on this topic is helpful.

• Practical strategies for trouble-shooting tier 2 & 3; case studies on how PBIS is being implemented in rural and low-resource schools

• effective coaching for PBIS

• District coaching/site coaching practices application, Academic data and behavioral data integration,

• I would love to hear what specific interventions and programs others are doing around the country.

• How to implement MTSS at the High School Level

• Systems to Support Teachers

• Next level of implementation of Restorative Practices and Trauma Informed Practices in PBIS

• Mindfulness in PBIS

• Mental Health and PBIS

• Funding for MTSS work

• Sustainability and fundraising opportunities

• more classroom implementation strategies

• Integrating trauma informed practices and PBIS

• Help with aligning DLT, specifically community partners

• Walking through the TFI

• District Alignment (supporting the recently released brief)

• Tier 3 Fidelity Tools

• District Coaching Support - policy/practice

• PBIS in Alternative Programs

• More RP, examples of circles etc.. recommended trainings for staff. addressing disproportionality.

Website In addition to the webinar materials the National TA center has provided a number of other documents and resources to all SCTG sites via http://www.pbis.org/sctg. The main SCTG webpage has received 29,786 views and 27,152 visits. The following table lists available resources and the number of downloads for each.

Document Name Number of Downloads

2014-2016 School Climate Transformation Grant Implementation Survey Summary

393

Six Month TA Status Report 234

Computer Applications for Monitoring Student Outcomes: Behavior

1041

Measuring Performance within School Climate Transformation Grants

1378

SCTG-Multi-tiered Behavior Framework Overview and TA Support

1137

SCTG Technical Assistance Worksheet 841

SCTG TA PBIS overview: Coordination and Capacity-Building Technical Assistance for Multi-tiered Behavioral Frameworks

888

SCTG and Technical Assistance: Frequently Asked Questions

880

Use of SCTG Funds for Incentives and Communication Products

592

SCTG Baseline Data Summary 663

Project Directors Meeting

The National TA center has co-facilitated an annual project directors meeting as a

pre-conference day aligned with the National Leadership Forum held in Chicago each

fall. Topics have included multi-tiered behavioral systems, integration and alignment,

equity and implementing advanced tiers. The table below summarizes the suggested

topics for future pre-conference days.

SEA LEA

• Using ESSA to sustain PBIS

• Integration of multiple initiatives; accessing, integrating, and funding community mental health supports in schools and as wraparound services

• update on funding/direction of DOE

• Helping the department and sites plan for Year 6--beyond the grant--sustaining all the work

• I'd like to be able to bring my Project AWARE Project Coordinator with me--our integration of the work depends on us cross-training

• More Tier II & III Behavior Support Strategies

• Best Practices for School Climate Walk-throughs

• Interconnected Systems Framework

• Evaluation support

• Tier II/III Models of implementation

• Fidelity of implementation

• Logistics of setting up District-wide data dig days. Behavior Education Policies

• Sustaining PBIS long term; What should PBIS recognition/acknowledgment look like in year 4;

• Scheduling and interventions at the secondary level

• Sustaining practices in years 4-5, turning practices over to in-district staff

• Tier 3 Systems Integrating with Health/Wellness/Academics

• Moving from only complex comprehensive FBA to Basic/Simple FBA

• Sustainability beyond the grant-what are districts doing and are there other funding sources?

• Building capacity and sustainability.

• Tier Two and Three Readiness

• Effective practices for Tier 2 and 3 supports/ interventions (which were at last year's conference, and we used much of what we learned!)

• BEST Practices for ways to share documents with sites, how to animate sites to participate when there are reluctant employees, how to move forward to improve culture when there are nay-sayers of new plans.

• Brief documents- (less wordy ones) for quick self eval, and some that include the check list of PBIS or Restorative features for schools & districts to have in place, quick on-line resource guides

• Sustainability

• Sustainability of systems with teacher and leadership turnover.

• Building district capacity for implementing MTBF

• More information about the Grant's fiscal year outlook. I feel like a large amount of time is spent on PBIS training--we are in Year 3 of the grant--please more forward and stop repeating the same information.

• Maybe divide into groups of returning project directors and new project directors--both groups need different information

• Possible Q & A session with strong SCTG implementers

• More Tier 2 and Tier 3 interventions

• Continue to offer the same

• Trauma Sensitive Schools

• Tier II Systems of Support

• Integrating Restorative Practices and PBIS would be nice. In addition, what is the possibility of sustaining this initiative if funding goes away!

• Transition after the Grant is over.

• Sustainability

• Grants

• Tier II Group Interventions - not an overview for beginners - an in-depth coverage of types of groups that are successful in keeping kids from rising to Tier III and/or getting kids back to Tier I

• Other Funding options for states/districts is SCTG funding ceases after year 3

• Sustainability and fundraising opportunities

• Some more evidenced-based Tier 2 strategies/building capacity at Tier 2

• Inclusive MTSS practices for ESE students

• What happens if you don't meet fidelity measures? How do you unpack when you don't meet requirements?

• Funding?

• Specific to HS

• It would be great if we could all submit some videos, pictures etc and "See" what other LEA's are doing, get a sense of how it has impacted students and ideas for sharing.

Summary and Next Steps

General Observations

Based on survey results from 2014- March 2017, we conclude with a few general

observations:

SEA

1. The SEAs have succeeded in establishing leadership, coordination and

stakeholder coordination

2. Recent improvement has occurred in collection of School Climate,

Implementation Fidelity, Student outcome, Disproportionality and Attendance

data

3. States have plans for implementation, but continue to need more development of

the technical support needed for improving depth of implementation (Tier II and

Tier III), Alignment of multiple initiatives, and Sustainability

4. All SEAs that responded to this survey are focused on implementing PBIS with at

least 50% also focused on Trauma Informed Care, Mental Health First Aid, and

Social Emotional Learning. Implementing multiple initiatives will be a major

area of focus.

LEA

1. The LEAs have made substantial improvement since 2014 on Leadership,

coordination, school team development, information systems, coaching capacity,

Tier I implementation and Launching Tier II/ Tier III implementation

2. Recent improvements have occurred in selection of Tier II and Tier III practices

for implementation, and development of district training and coaching capacity.

3. Districts have become much more focused on the collection and use of data, and

are targeting strategies for improving either the type of data available to schools,

or the process for using existing data sources.

4. All districts that responded to this survey are engaged in implementing PBIS, and

at least 50% of LEAs area also focused on Restorative Practice Trauma Informed

Care, Mental Health First Aid and/or Social Emotional Learning

5. The LEAs have made major gains in the quality and array of evaluation data they

are collecting (school climate, fidelity, student outcomes, SPED referrals,

Disproportionality, Attendance)

Technical Assistance

1. Technical Assistance is being directed primarily to SEA and LEA leadership

teams and Coordinators. Most of the TA events are 1 hour to 1 full day in length

and involve either Individual or On-site meetings.

2. Webinars and implementation materials continue to be accessed, and used, but the

participation is becoming more targeted to sites specifically focused on the

content of webinars and materials.

Next Steps

The TA center is using survey data across sites to inform the selection of webinar

and conference session topics. Individual TA providers are using survey data to support

the development of individualized action plans within sites. In addition to informing the

organization of TA supports, the implementation survey will be used to track the progress

of LEAs and SEAs as they work to develop MTBF implementation capacity.