Using Data Unique to High Schools

Susan Barrett sbarrett@pbismaryland.org

Enduring Understanding:

Know how to you use data to plan for interventions – at the beginning of the year and on-going

2 Views!!!Student LevelSystems Level

Essential Questions

• How can teams convert data to a plan of action based on fidelity and outcomes?

• How can team make decisions with data beyond discipline?

Preparing with Data

1. Gathering and Action Planning2. Organizing and Implementing3. Screening4. Linked to Outcomes5. Celebrating and Social Marketing6. Continuous Improvement and Planning

Career Builder

Gathering and Action Planning

• Discipline Data Review Problem Behavior, Time, Location, frequency, grade, ethnicity, teacher• Surveys and Focus Groups• Engagement• Attendance• Credit Accrual, GPA, PSSA/Keystone Exams

Top 3 Minor Infractions by Grade Level Per 100 Students

0

0.5

1

1.5

2

2.5

3

3.5

Fresh Soph Juniors Seniors

Grade Level

# o

f R

efe

rra

ls P

er

10

0 S

tud

en

ts

1-4 Classroom Disruption

1-3 Dress code

1-6 Tardy to class

1-4 Classroom Disruption

1-3 Dress code

1-6 Tardy to class

1-3 Dress code

1-4 Classroom Disruption

1-6 Tardy to class

1-3 Dress code

1-4 Classroom Disruption

1-2 Leaving Class

Top 3 Reasons for Major Referrals Per Every 100 Students

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Fresh Soph Juniors Seniors

Grade Level

# o

f R

efe

rra

ls P

er

10

0 S

tud

en

ts

2-6 Profanity

2-5 Failing to Abide Rules

2-8 Defy Authority

2-6 Profanity

2-5 Failing to Abide Rules

2-8 Defy Authority2-6 Profanity

2-5 Failing to Abide Rules

2-8 Defy Authority

2-6 Profanity

2-5 Failing to Abide Rules

2-8 Defy Authority

MDS3 School Climate SurveySAFETY•Physical Safety - I feel safe at this school.•Bullying - During the current school year have you been bullied? •Social-Emotional Wellbeing - It is OK to hit someone if they hit me first.•Substance Use - During the past 30 days, how many days did you have 1 drink of alcohol?ENGAGEMENT•Relationships and Connectedness - I feel like I belong.•School Participation and Academic Emphasis - I like this school.•Parental Involvement - This school tries to involve parents or guardians.•Culture of Inclusion and Equity - Teachers believe that all students can do well if they try.ENVIRONMENT•Order and Discipline - Disruptions by other students can get in the way of my learning.•Physical Environment - The school has a bright and pleasant appearance.•Supportive Services - This school has programs to deal with violence and conflict among students.

Results from MDS3 Spring 2013 (31 intervention schools, N = 14,150)

Safety• 81.0% feel safe at school• 22.4% reported being bullied during the school year• 41.0% reported that other students try to stop bullying • 21.5% often or very often feel sad• 81.6% reported that substance use is a problem at their schoolEngagement• 66.5% feel they belong at school• 80.2% feel their teachers encourage them to work hard• 45.0% feel their parents are informed when they do well• 60.9% feel students of all races are treated equallyEnvironment• 73.7% report that disruptions in the classroom get in the way of their

learning• 47.3% report the school building is clean and well-maintained• 68.4% report that students who need help with their problems are able

to get it at school

“I feel like I belong”

20,619 students6,804 disagree or strongly disagree with this statement

Focus Groups

• What does a referral mean to you? How do you feel when you get a referral?– Don’t care – they be like you’re getting a referral

and I’m like your point is…|– Feel like the teachers write half the truth and half

lies– If I get one, I get one – it depends if it’s for

something dumb like not sitting in your seat.

Learning Preferences• I like to have new concepts

modeled• Enjoy working in groups• I often get the “big” picture

and have to work to note the details

Accommodations that Work• Extended time• Help from my friends• Working with people with

different learning styles

I Want You to Know• I don’t like timelines• I work hard• I play hard

Interests• Animals• Cooking• Outdoors

Name: John McNaughtAddress: Rabbits Foot RdHinton, VA DOB: 01-07-75

You’re InvitedDate: 08-04-09 Time: 10:00 am

Site Visits

• Partially funded by William T. Grant Foundation (PI:Bradshaw)

– 25 classrooms per school (≈1500 per time point)– 30+ non-classroom locations

• Using handheld devices to collect data• Instruments

– Assessing School Settings: Interactions of Students & Teachers (ASSIST): Rusby et al. (2001); Cash & Debnam

• Praise, opportunities to respond, punishing statements, transitions, supervision, positive interactions, engagement, aggressive behavior etc.

• Both event based and global ratings– School Assessment for Environmental Typology (SAfETy):

Bradshaw, Lindstrom Johnson, Milam, & Furr-Holden• Features of the school environment that encourage access control,

surveillance, territoriality, physical maintenance, and behavioral management (e.g., disorder, substance use, broken windows)

2. Organizing and Tracking for Fidelity

• Benchmarks of Quality• School wide Evaluation Tool• Team Implementation Checklist

SET/ISSET

• The School Wide Evaluation Tool (SET) and Individual Student Systems Evaluation Tool (ISSET) document the types of programs schools are using to support student behavior and learning– Covers a range of programs

• The SET/ISSET includes brief interviews with an administrator, the student support team leader, random staff and students, observations, and a review of specific school documents

Fidelity Data: SET

Expectations Defined

Behavioral Expectations

Taught

On-Going Reward Sys-

tem

System for Responding

to Behavioral Violations

Monitoring and Decision

Making

Management District Support

Total SET score

0

20

40

60

80

100

81.076.9

63.2

94.191.8

86.682.8 82.4

Year 3

Imp

lem

enta

tion

%

Fidelity Data: ISSET

Foundations School-wide In-terventions

Targeted In-terventions

Intensive Indi-vidualized Inter-

ventions

Total I-SSET score

0.0

20.0

40.0

60.0

80.0

100.0

77.0

63.7

81.2

92.2

78.5

Year 3

Imp

lem

enta

tion

%

3. Screening

Using the Referrals by Student report as a Universal Screening Tool

24

26

Definitions of Problem Behaviors

• All problem behaviors are covered and none of the definitions overlap

• Consistent definitions make data collection much more accurate and reliable

• The addition of minor problem behaviors assists in the summary of minor infractions

• Agree on behavioral definitions• Brainstorm classroom vs. office managed behaviors• Agree on behaviors to list on office referral

Nuts and Bolts

T- Chart

Staff Managed or Minor Problem Behaviors

• Eating, drinking, chewing gum• Disruption• Horseplay• Defiance to another student• Pushing or shoving• Lying/cheating• Public Display of Affection• Writing on School Property• Disrespect, minor to another

student or another student’s belongings

Office Managed orMajor Problem Behaviors

• Defiance/Disrespect/Non-Compliant

• Abusive or inappropriate Language

• Fighting or Physical Aggression• Disruption• Theft/Forgery• Property Damage/Vandalism• Use or Possession of

Drugs/Alcohol

Develop a Process Flow Chart• Create a flow chart to guide student behavior

management

• Keep it as simple as possible

• Keep it as clear as possible

Observe Problem Behavior

Warning/Conference with Student

Use Classroom Consequence

Complete Minor Incident Report

Does student have 3 MIR slips

for the same behavior in the same quarter

•Preparedness•Calling Out•Classroom Disruption•Refusal to Follow a Reasonable Request (Insubordination)•Failure to Serve a Detention•Put Downs•Refusing to Work•Inappropriate Tone/Attitude•Electronic Devices•Inappropriate Comments•Food or Drink

•Weapons•Fighting or Aggressive Physical Contact•Chronic Minor Infractions•Aggressive Language•Threats•Harassment of Student or Teacher•Truancy/Cut Class•Smoking•Vandalism•Alcohol•Drugs•Gambling•Dress Code•Cheating•Not w/ Class During Emergency•Leaving School Grounds•Foul Language at Student/Staff

Write referral to office

Administrator determines

consequence

Administrator follows through

on consequence

Administrator provides teacher

feedback

Write the student a

REFERRAL to the main office

•Issue slip when student does not respond to pre-correction, re-direction, or verbal warning

•Once written, file a copy with administrator

•Take concrete action to correct behavior (i.e. assign detention, complete behavior reflection writing, seat change)

SIDE BAR on Minor Inc ident Repor t s

•Issue slip when student does not respond to pre-correction, re-direction, or verbal warning

•Once written, file a copy with administrator

•Take concrete action to correct behavior (i.e. assign detention, complete behavior reflection writing, seat change)

SIDE BAR on Minor Inc ident Repor t s

Is behavior office

managed?

ClassroomManaged

Office Managed

No Yes

Define Behavior Expectations

Model Appropriate Behavior

Observe Problem Behavior

Decide: Is the behavior office managed?

Write referral to the office

Use the following teacher consequences: (Must be documented prior to writing office referral for Teacher Managed Behaviors.) Parent contact is a MUST!

1st Offense: Student/Teacher Conference (Warning)

2nd Offense: Teacher determined consequence (i.e. lunch or after-school detention, seat change, loss of privilege)

3rd Offense: Teacher determined consequence + Parent Contact

4th Offense: Office Referral

Teacher Managed Behaviors Office Managed Behaviors

•Language•Minor Insubordination•Lateness•Calling Out•Minor Throwing•Non-Performance or Refusing to Work•Preparedness•Minor Dishonesty•Disrespect•Disruption•Inappropriate Tone or Attitude•Dress Code Violation (send to the office)•Food or Drink•Gum•Running in the Halls/Horseplay

•Chronic Minor Infractions (x3)•Failure to Comply with Teacher Consequences•Cell Phone(confiscate)•Fighting or Aggressive Physical Contact•Profanity Towards Teacher•Major Insubordination•Weapons•Harassment•Bullying•Academic Dishonesty•Skipping Class•Drugs/Alcohol•Smoking•Vandalism•Theft• Threats•Gambling•PDA

Administrator Restates Behavior Expectation with Student

Administrator Issues Appropriate Consequence

Administrator Contacts Parents

Administrator Provides Teacher Feedback

+

+

+

NO YES

Per Marking Period

Total Office Discipline Referrals (ODR) per Month

LOOKS LIKE OCTOBER WAS A BUSY MONTH

Avg. # ODR/Day/Month

Consider # of SCHOOL DAYS in the month.

0

500

1000

1500

2000

2500

3000

94-95 95-96 96-97 97-98 98-99 99-00 00-01 01-02 02-03 03-04 04-05 05-06

Tota

l ODR

s

Academic Years

FRMS Total Office Discipline Referrals

SUSTAINED IMPACTPre

Post

How long would it take to answer the BIG 5 SW discipline questions in your school?

1. Who exhibited the problem behavior ?

2. What did they do?3. Where did they do it?4. When did they do it? 5. Is this a common error?

# Referrals by Student

# Referals by Problem Behavior

# Referrals by Location

Combined Data Using Vlookup in Excel

http://www.act.org/explore/norms/spring8.html

See YouTube examples: http://www.youtube.com/watch?v=wH6jPVHnc9Q

Recognize and RespondWhen do we kick it up a notch?Would you notice when something was up?Family and Community Connections360 View: Multiple Data Points as Natural Screeners- LOW Threshold!!

– Attendance– Office Referral– Suspension– Instructional Time– Tardy– HW Completion– GPA– Credit Accrual

• Youth has 2 Major ODRs• Youth has 1 Suspension• Youth experiences more than ? minutes out of instruction • Youth misses more than ? days unexcused absences• Youth drops GPA by more than ??• Youth – benchmark testing- McIntosh • Youth- incomplete class work/homework• Attendance (look at predictors for drop-out and school completion)• Admin Referral• Teacher/Staff Referral• Family Referral• Other:

Decision Rules for Access to Advanced Tiers (and decision rules for prevention-if we can predict the trajectories , then we can prevent it

from happening)

Early Warning Indicators Course Performance inCore Subjects GPA

Credits FCAT Attendance

Office Discipline Referrals

AdditionalFactors

On-Track Indicators

On-Track

Meeting all graduation requirements Cs or better in all areas

2.5 or more Meeting credit graduation requirement for grad plan year

Level 3 or above or concordant scores within the same school year

4% or less absences per quarter or semester

3 or less Level I and/or minor referrals

DisengagementNo extra curricular involvementSubstance AbuseHigh MobilityMental health issuesFree/Reduced lunchFoster/group homeTransient/HomelessParent unemployment Student employmentChanges in behavior/ appearance More recent traumatic eventMissed guidance appointmentsNo show for yearbook picture

At-Risk forOff Track

Lacking 1 graduation requirement

2.0 to 2.49 Behind 1 Credits

Level 2 on FCAT

5% or more absences per quarter or semester

4 or less Level I and/or minor referralsLevel II ODRs per semester

Off-Track

Lacking 2 graduation requirementsFailing 1-3 classes

Less than 2.0 Behind 3 credits Not passed both sections of 10th grade FCAT or retakesNo concordant scores

10% absences per quarter or semester

5 or more Level I and/or Level II ODRs per semester

Highly Off-Track

Lacking 2 or more graduation requirementsCurrently failing 3 or more classes

Less than or equal to 1.5

Behind 4 or more credits

Not passed 10th grade FCAT or retakesNo concordant scores

15% or more absences per quarter or semester

5 or more Level II ODRs for fighting/ profanity/ disruption per semester

ExtremelyOff-Track

Meeting no graduation requirements2-3 Years Behind

Less than or equal to 1.0

Not meeting cohort graduation plan

Not passed 10th grade FCAT or retakesNo concordant scores

20% or more absences per quarter or semester

Established pattern of severe behavior Level II & III ODRs

Name Grade Level

GPA last year

Behavior Referrals

Core Grades

Attendance Credits

Jana 9 2.2 4 ref 1 D 1 F 82% On TrackBlake 11 1.3 0 3F 88% -3Toby 10 2.8 16 ref 1 S 2D 1 F 84% -2Carlos 10 .7 22 S 3 S 4 F 62% -4Yvonne 9 2.7 2 ref 2 D 1 F 86% -1Lin 11 2.3 0 1 F 90% -2Maria 12 1.9 16 ref 2 S 4 D 1 F 74% -4Doug 9 3.1 2 ref 1 F 81% On TrackTyrone 9 2.9 10 ref 2 D 89% On TrackSam 10 2.4 13 ref 2D 1 F 87% -2Paul 9 3.4 1 ref 1 D 86% On TrackTia 9 3.7 0 2 C 60% On Track

Activity: Student List

Who gets access to an intervention that integrates academic/behavioral support ? Choose 6 students.

Focus Areas

1. Freshmen Support2. On Time Graduation

State Leadership Team Organized by 2 Focus Areas

PBIS OTGR Intervention Team:What are this student’s obstacles to

making it to graduation?What is the plan to help him succeed?

• 18 Years Old• Failing Algebra at Semester• Absent 20% of school days

so far• Needs English HSA• Homeless-living w/friend

BIRD HS - OTGR Data DashboardStudent Name MENTOR % Ab

SOL's Needed SPED ESOL WIA Notes Failing 1st Nine Weeks

A Oneill 5 2 4% 5-Verified CreditsNo Yes No 11th-Needs 2 more classesB Clarke 5 3 16% None No No No Govt-Smith,Physics-AustenC Lythgoe 5 2% None No No YES English 12-OccidentD Graham/Diebel 5 12% Writing Yes NO ? OSSE Joseph 5 4 2% None No No No Drop in fair 4 ClassesF Hoffner 5 0% None No No No Afr Am Lit-Occident, Physics-AustenG Hackett 5 2 7% RLR No No No Eng 12-BovaH Lau 5 11% RLR No No No Eng 12-BovaI Goehle 5 26% None No No No Truancy ProgramEng 12-Howsmon, Art 1-MorseJ Gerlach 5 18% None No No no Drop in fair/Success ?4 ClassesH Mooberry 5 3 22% None No No No Govt-Richmond, Afr Am Lit-OccidentJ Brown, C 5 3 21% None No No No Baby Eng 12K Simon 5 26% None No No No OSSL Kistner 5 3 33% None No No ? Entered 10/24Mktg, Eng 12-Occident, Govt-RichmondM Beymer/Davis/Vega 5 9% None Yes No No Comp Math, Physics-AustenN Luqman 5 4 2 11% Writing No Yes No Govt-Richmond, Alg 2-Ligatti, EO Bova 5 4 3 19% None No No No Eng 11, Eng 12-Bova, CTCP Mudd 5 4 3 44% None No No No Truancy ProgramComp Math, Geom-Schenk, Govt-Stoll, CTCQ Westcott/Reeves 5 4 3 25% RLR No No No SOL Academy5 classesR Nicolas 5 4 5% None No Yes NO Geom-Martin, Eng 12-BovaS Waller, C 5 2 9% Writing No No NoT Brown, B/Kalman 5 4 3 16% None No No No Govt-Smith, MrktU Covington 5 2 5% Needs 3 No NO No SOL AcademyAFDA-WierschemV Hinton 5 2 0% Writing & RLRNo Yes NOW Barker 5 4 5% None No No NO Afr Am Lit-Occident, Govt-SmithX Oshea 5 2 0% Writing & RLRNo No YES SOL AcademyY Brock 5 4 11% None Yes No NO Afr Am Lit-OccidentZ Richmond 5 2 16% Earth Sciene & WritingNO Yes NO Physics-Austen, Govt-RichmondAA Gocha 5 4 2 11% Writing, RLRYes No YES Afr Am Lit-Occident, Govt-HintonBB Hetterich 5 4 5% None No no no 5th Year SeniorComp Math, Eng 11-MannsCC Arcaro 5 4 3 30% None No No No Truancy ProgramEng 12-Occident, Govt-Richmond, Chem-ClarkeDD Louis 5 4 2 11% Writing, RLRNo Yes Yes SOL AcademyPhysics-AustenEE Henderson 5 4 7% Writing & RLRNo No No 504 Health/PE 2-Henderson, Eng 12-Bolton, Eng 11-OsheaFF Bakke 5 4 0% None No No No Eng 12-Howsmon, Govt-HintonGG 1 32% 1-Hist 1-Math,RLRNo No No Earth Sci 2, Eng 12-HowsmonHH White 5 4 7% None no no no Ecology-Layne, Eng 12-Bolton, Us Hist-Hailstock

5=WDYSF Mentor4=Grade Conf. 3=Attendance Conf.2=SOL Plan

1st quarter 2nd quarter 3rd quarter 4th quarter

Charger Expectations

Life Long LearningHonor & Respect

Success&

Life Skills

MathContinue

2nd quarter skills class

Pick up 2nd skills classLiteracy

LeadershipPeer Ambassador

Behavioral SupportSocial/Emotional

Graduate to Leadership

48

Freshman Success

Three times over the following three weeks, ask students (can happen in any class) questions about the graduation requirements – this should happen very quickly and take only 3 5 minutes. ‐Clarify any incorrect information and remind students to check their progress regularly and where to find thegraduation requirements in full (student handbook, school website, etc.) Sample questions may include:

49

Sample Questions

1. How many credits do you have to earn to become a sophomore?

2. How many credits will you earn for [this, Math, History] class if you earn a C or better?

3. What is the minimum grade you can in a class to still earn credit for it?

4. How many credits do you need to graduate?5. What else do you need to do to graduate

besides earn credits for classes?

Link to Outcomes

Celebration and Social Marketing

• Buy and Support • Sustaining and Expanding

• Tardy Project Example

Building the Case for Support

• Linking Coach role to impact

Tracking Support

Organizational/Systems Change OR The Development of Effective, Productive,

Collaborative Systems that create optimal Conditions for Learning

Systems Change Coach

(Facilitator/Communicato

r)

Support to Leadership

Skill Coach for

Practitioners

0

500

1000

1500

2000

2500

3000

89.425 115.67548.65

1206.67

553.449999999999 515.99

2529.86

1.25 29 11.55

354.8775

116.65200.6825

714.01

3.7587

34.65

1064.6325

349.95

602.047499999999

2142.03

Year1

Year2

y2_projected

56

Data Review

• What do you need from District?• Do you have vertical team that provides info

about 8th grade performance information?• What are some natural ways freshmen can

access existing supports?• How can you communicate so support can

happen in all classes?• Can some students get a class ?• Syllabus? Check in, Retest Request?