ies summer research institute: single-case intervention design and analysis day 2 8:45-10:15 rob...

IES SUMMER RESEARCH INSTITUTE:SINGLE-CASE INTERVENTION DESIGN AND ANALYSIS

Day 28:45-10:15Rob Horner

Overview of Complex and UniqueSingle-case Designs

Complex Designs

Goals and Assumptions

• Assumption: • Fluent with ABAB, Multiple Baseline and

Alternating Treatment Designs

• Objectives• Define features of Changing Criterion, and

Multiple-Probe Designs

• Apply Single-case Design and Analysis logic to construction of more complex design challenges.

Defining Features of Changing Criterion Designs• Within subject analysis• Independent variable needs to have at least four levels

• Three demonstrations of effect

• Document baseline performance with first IV level• Change the level of the IV and monitor change in DV

• Immediacy of change important• Absence of trend and variability important

• Repeat level (criterion) change in IV two more times.• Analysis involves same standard as other single-case

designs: Three demonstrations of effect across three different points in time.

Changing Criterion Design

0

5

10

15

20

25

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Days

Occ

urr

ence

s o

f P

rob

lem

Beh

avio

r

BL: No Reinf Reinf < 17 Reinf < 12 Reinf < 5Independent Variable: Schedule of Reinf Level 1: No Reinf Level 2: Reinf for fewer than 17 events Level 3: Reinf for fewer than 12 events Level 4: Reinf for fewer than 5 events.

First Demonstration of Effect

Second Demonstration of Effect

Third Demonstration of Effect

Research Question: Is there a functional relation between contingent delivery of reinforcement for reduced problems behavior and reduction in the level of problem behavior?

Changing Criterion Analysis: 1. Stable within phase patterns 3. Five data points per phase2. Immediacy of effect


0

5

10

15

20

25

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Days

Occ

urr

ence

s o

f P

rob

lem

Beh

avio

r

BL: No Reinf Reinf < 17 Reinf < 12 Reinf < 5

No Demonstration of Experimental Control

First Demonstration is Okay

Second Demonstration is compromised by trend

Third Demonstration is compromised by trend and floor effect


0

5

10

15

20

25

Days

Occ

urr

ence

s o

f P

rob

lem

Beh

avio

r

BL: No Reinf Reinf < 17 Reinf < 12 Reinf < 5

No Demonstration of Experimental Control

Would you use a Changing Criterion Design?If “yes” draw the design with expected data.

• Is there a functional relation between defining self-management criteria for smoking (20, 15, 10, 5) and reduction in the number of cigarettes smoked per day?

• Is there a functional relation between number of pages of reading needed to achieve free time, and the number of words read per reading session?

• Is there a functional relation between use of FCT and reduction in the level of problem behavior?

• Is there a functional relation between use of Ritalin versus Risperdal, and reduction in rate of stereotypy?


0

5

10

15

20

25

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29

Days

Occ

urr

ence

s o

f P

rob

lem

Beh

avio

r

BL: 20 Cig 15 Cig 10 Ci g 5 Cig

Cig

aret

tes

per

day

Activity:

• Propose a new research question that would be appropriately addressed via a changing criterion design

• Propose a research question that would NOT be appropriately addressed via a changing criterion design

MULTIPLE PROBE DESIGNS

Multiple Probe Design: A Variation of Multiple Baseline

• Multiple Probe Design:• A variation of the Multiple Baseline Design in

which baseline data are probed at different points in time rather than monitored continuously.

• Probe Data are always collected at the beginning of BL and just before Intervention.

• Probe Data may also be collected at points of intervention in other series within the design.

Conditions when Multiple-probe is Used

• Data collection is viewed as intrusive/expensive, and initial Baseline data document a stable pattern.

Standards for Multiple-Probe

• Meet Single Subject Design Standards• 5 BL data points• Active manipulation of IV• At least three demonstrations of effect

Par

tici

pant

1P

arti

cipa

nt 2

Par

tici

pant

3P

arti

cipa

nt 4

Dep

ende

nt V

aria

ble

Baseline Intervention

Days/Weeks/Months/Sessions

Non-concurrentMultiple Baseline

A BA B

A B

A B

A B

Student 1 Student 1

Student 1

Student 2 Student 2

Student 2

Student 3 Student 3

Student 3

1 2

3

Actual Time

Non-concurrent Multiple Baseline

Time

Meets Evidence Standards

Meets Evidence Standards with Reservations

Meet standards for a multiple baseline design

Meets standards Meets standards with reservations

Overlapping initial pre-intervention sessions (Baseline)

Each case has probe points in the initial three sessions

Each case has at least one probe point in the first three sessions

Probes prior to introducing the independent variable to a given case

Three consecutive points just prior to introducing the independent variable and one probe point every eight sessions

At least one point just prior to introducing the independent variable and one probe point every eight sessions

Probes for subsequent cases when introducing the independent variable to a preceding case

At least one point placed either immediately prior to the first intervention session for the earlier case or once the intervention criterion is reached for the earlier case.

At least one point placed either immediately prior to the first intervention session for the earlier case or once the intervention criterion is reached for the earlier case.

WWC Standards for Multiple Probe Designs

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

20

40

60

80

100

Parti

cipan

t 1

Probe Intervention

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

20

40

60

80

100

Parti

cipan

t 2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

20

40

60

80

100

Session

Parti

cipan

t 3

Rate

of D

isen

gage

men

t

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

20

40

60

80

100

Parti

cipan

t 1

Probe Intervention

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

20

40

60

80

100

Parti

cipan

t 2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

20

40

60

80

100

Session

Parti

cipan

t 3

Rate

of D

isen

gage

men

tMeets Design Standard

Five BL data points for all series (3 overlap)

Three Pre-intervention data points per series

At least one data point per 8 BL sessions

At least one data point per series for Prior-intervention series

Meets WWC Standards

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

20

40

60

80

100

Parti

cipan

t 1

Probe Intervention

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

20

40

60

80

100

Parti

cipan

t 2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

20

40

60

80

100

Session

Part

cipan

t 3

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

20

40

60

80

100

Parti

cipan

t 1

Probe Intervention

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

20

40

60

80

100

Parti

cipan

t 2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 160

20

40

60

80

100

Session

Part

cipan

t 3

Meets Design Standards with reservation

One common BL data point per series

At least one Pre-Inter data point per series

At least one data point per 8 BL sessions

At least one data point per series for Prior-intervention series

Meets WWC StandardsWith Reservation

1 2 3 4 5 6 7 8 9 10 11 120

20

40

60

80

100

Parti

cipan

t 1

Probe Intervention

1 2 3 4 5 6 7 8 9 10 11 120

20

40

60

80

100

Parti

cipan

t 2

1 2 3 4 5 6 7 8 9 10 11 120

20

40

60

80

100

Session

Parti

cipan

t 3

Does NOT Meet WWC Standards

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 260

20

40

60

80

100

John

- Se

t 3 W

ords

Probe 1 Int. Probe 3Int. Int. Probe 4Probe 2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 260

20

40

60

80

100

John

- Se

t 2 W

ords

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 260

20

40

60

80

100

Session

John

- Se

t 1 W

ords

Percent Correct in Reading

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 260

20

40

60

80

100

John

- Se

t 3 W

ords

Probe 1 Int. Probe 3Int. Int. Probe 4Probe 2

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 260

20

40

60

80

100

John

- Se

t 2 W

ords

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 260

20

40

60

80

100

Session

John

- Se

t 1 W

ords

Percent Correct in Reading

Meets WWC Design Std

Considerations for Multiple Probe

• Decreases the logical and fiscal cost of data collection across long Baselines.

• Should NOT be viewed as a strong control for measurement effects (Instrumentation… number of times participant experiences measurement).

• Number of data points per “probe”• WWC standard• Issues for consideration in the field

Increasing Complexity in Single-Case Designs

For each study

• Define the research question(s)

• Determine if the design allows assessment of functional relation

• Determine if the data within the design document a functional relation (per research question)

EVALUATION OF A CAFETERIA SOCIAL SKILLS PROGRAM IN AN ELEMENTARY SCHOOL IMPLEMENTING SCHOOL-WIDE PBS

Jennifer Jeffery

0

1020

30

4050

60

70

8090

100

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41

0

10

20

30

40

50

60

70

80

90

100

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41

0

10

20

30

40

50

60

70

80

90

100

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41

Pro

po

rtio

n o

f In

terv

als

wit

h

Dis

rup

tive

Beh

avio

r

Sessions

Teaching Session 1

Teaching Session 2

Teaching Session 3

Group 1

Group 2

Group 3

Aide Training

Baseline Cafeteria Social Skills Program

Figure 5. Effect of the Cafeteria Social Skills Program on disruptive behaviors in the cafeteria.

Jennifer Jeffery

• Intervention package• Breaks in timeline• Measure

A

Assigned Lunch Tables

Group 1 Group 2 Group 3

THE EFFECTIVENESS OF INTERVENTION STRATEGIES BASED ON FUNCTIONAL BEHAVIORAL ASSESSMENT.

Kimberly L. Ingram, Teri Lewis-Palmer and George SugaiUniversity of Oregon,

Question of Interest

• Is there a decrease in the level of problem behavior when a behavior support plan is based on behavioral function?

• Conduct a functional assessment:

• Plan A: Indicated by functional assessment• Plan B: Contra-indicated by functional

assessment

0

10

20

30

40

50

60

70

80

90

100

1 3 5 7 9 11 13 15 17 19 21 23 25 27

% In

terv

als

w/ P

.B.

Sessions

% Intervals w/ P.B. for Carter

Baseline IndicatedIndicated Indicated Modified

Contra-Indicated

Contra-Indicated

Ingram et al

A B C B C B’

FUNCTION-BASE SUPPORT BY TYPICAL SCHOOL BEHAVIOR SUPPORT TEAMS

Melissa K. BergstromUniversity of Oregon

Research Questions

• Is there a functional relation between implementation of function-based support by independent school teams and reduction in the level of student problem behavior?

• Are the hypothesis statements developed by school teams valid?

• Is function-based support in the school context feasible (i.e. acceptable, efficient, completed with integrity)?

Melissa Bergstrom

EFFECTS OF THE BEHAVIOR EDUCATION PROGRAM ON PROBLEM BEHAVIOR IN MIDDLE SCHOOL

Leanne HawkenUniversity of Oregon

Leanne Hawken

USING FCT TO ACHIEVE GENERALIZED REDUCTION OF PROBLEM BEHAVIOR

Holly Reed Schindler and Rob Horner

Three Research Questions

Main Research Question• Is there a functional relationship between reduction in the

level of problem behavior in the secondary settings and implementation of a Trans-situational Intervention (TSI)?

Additional Research Questions2) Is there a functional relationship between reduction in

level of problem behavior in the primary implementation setting and implementation of a TSI?

3) Are TSI procedures rated as efficient and socially acceptable?

Critical Features of Design

• Document problem behavior in primary setting during Baseline.

• Document problem behavior in secondary settings with introduction of low effort intervention.

• (Low Effort alone is ineffective)

• Document that introduction of Intervention in primary setting is associated with:

• Reduction of problem behavior in primary setting • No change in problem behavior in secondary settings

• (TSI alone does not produce change in secondary settings)

• Document that introduction of TSI + Low Effort is associated with reduction of problem behavior in secondary settings.

Holly Reed Schindler

The Second Grade Project

Sarah Fairbanks & George Sugai University of Connecticut David GuardinoUniversity of OregonMargaret LathropBethel School District

Study 1: Independent Variable- Check-in & Check-out Intervention • Based on Hawken & Horner’s (2003) model• Participants carried behavior cards throughout the day.• Teachers assessed points after each time block.• Points were added up at the end of the day. If participants

earned a certain percentage of points (range, 70% to 90%) the whole class earned a reward (e.g., extra recess, class game).

• Intervention included frequent and regular opportunities for feedback

• Non-function based group intervention

0

10

20

30

40

50

60

70

80

90

100 BL CI/CO

CI/CO +75%

CI/CO +80%

CI/CO +90%

Helena

School Days

Per

cen

t of

Int

erva

ls E

nga

ged

in P

robl

em

B

ehav

ior

0

10

20

30

40

50

60

70

80

90

100

Jade

0

10

20

30

40

50

60

70

80

90

100

Farrell

Began meds.

Class B Results

Descriptive not experimental

Fairbanks et al., Study 2

• For students who did not respond to CICO, is there a functional relation between delivery of function-based interventions, and reduction in level of problem behavior?

0

10

20

30

40

50

60

70

80

90

100

Ben

0

10

20

30

40

50

60

70

80

90

100

Marcellus

BL CI/CO

CI/CO75%

CI/CO80%

FB plan

FB plan 2

0

10

20

30

40

50

60

70

80

90

100

Blair

0

10

20

30

40

50

60

70

80

90

100

Olivia

Per

cen

t of

Int

erva

ls E

nga

ged

in P

robl

em

Beh

avi

or

Study 2 Results

School Days

Fidelity of First Step to Success Implementation

Billie Jo RodriguezSheldon LomanRob Horner

Research Question

• The purpose of the study was to assess the impact of incorporating a specific performance feedback procedure (coach-provided feedback to teacher) within the design of FSS intervention protocols. The primary research question asked if there was a functional relation between the implementation of coach-provided performance feedback package and level of teacher fidelity in implementation of FSS. A secondary research question focused on the relation between teacher fidelity of FSS implementation and sustained reduction in level of student problem behavior.

1. Is there a functional relation between CF and change in problem behavior?

2. Is there a functional relation between CF and FSS (card) fidelity?

THE EFFICACY OF TRAINING SCHOOL PERSONNEL TO BUILD BEHAVIORAL INTERVENTIONS FROM FUNCTIONAL BEHAVIORAL ASSESSMENT INFORMATION

Kathleen Strickland-Cohen

Building Local Behavioral Capacity

• Two Main Ideas

• Behavior support is more effective and efficient when it is built from an FBA.

• Functional behavioral assessments can be done accurately and efficiently by typical building personnel for “BASIC” problems.

Question of Interest• Is there a functional relation between implementation of a

behavior support plan by a Team Lead who has received Basic BSP Development training, and reduction in the level of problem behavior by students?

Basic FBA to BSP Trainer’s Manual Sheldon Loman, Ph.D. Portland State University

M. Kathleen Strickland-Cohen, Ph.D. University of Oregon

Chris Borgmeier, Ph.D. Portland State University

Robert Horner, Ph.D. University of Oregon

Defining & Understanding Behavior FBA: Conducting FBA Interviews

FBA: Observing & Summarizing Behavior

Critical Features of Behavior Support Plans * Core elements

Building Behavior Support Plans from FBA Information* Technically Sound WITH Contextual Fit

Implementation and Evaluation Planning

Leading a Team through the Behavior Support Planning Process

Instructional Content

Many School Personnel are Skilled at Collecting FBA Information but NOT skilled at identifying if behavior support plan elements are consistent with the FBA

Participant Pre Test Post Test Percent Change

163% (A)

96% (B) +33%

267% (A)

84% (B) +17%

369% (A)

94% (B) +25%

465% (A)

86% (B) +21%

560% (A)

88% (B) +28%

663% (A)

90% (B) +27%

743% (A)

82% (B) +39%

861% (B)

92% (A) +31%

963% (B)

82% (A) +19%

1045% (B)

80% (A) +35%

1167% (B)

90% (A) +23%

1261% (B)

86% (A) +25%

1380% (B)

94% (A) +14%

Mean 62% 88% +26%

Pre/Post-Test Results: Assessment of BSP Knowledge

Sebastian

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27

Off-task

Talk outs

Implementation Fidelity

Bailey

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27

Off-task

Int w/ PB


Micah

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27

Int with PB

ImplementationFidelity

Charlie

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27

Int with PB


Gareth

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27

Int with PB


% 10 sec intervals

Sessions

Baseline Implementation of Function-based BSP

Non-concurrent

Multiple Baseline

Fidelity/ Problem Behavior

Strickland-Cohen & Horner

Sebastian

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27


Engagement

Bailey

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27


Engagement

Micah

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27

ImplementationFidelity

Engagement

Charlie

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27


Engagement

Gareth

0

25

50

75

100

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 2 21 2 2 2 25 2 27


Engagement

Sessions

% 10 sec intervals

Baseline Implementation of Function –based BSP

Strickland-Cohen & Horner

Non-concurrent

Multiple Baseline

Fidelity/ Academic

Engagement

District Field-Test

• Sustained use of Basic FBA-BSP approach

• District effectiveness at “whole district” implementation.

Implications

• If you are the behavior specialist for a district:

• 1. Continue to provide direct assistance for complex behavior support cases.

• 2. Use the Basic FBA and BSP materials to train building-level personnel to do “BASIC” FBA and BSP development.

• (www.pbis.org)

Day, H. M., Horner, R. H., & O'Neill, R. E. (1994). Multiple functions of problem behaviors: Assessment and intervention. Journal of Applied Behavior Analysis, 27, 279-289.

Adapting Single Case Designs

0%

10%

20%

30%

40%

50%

60%

70%

1 3 5 7 9 11

13

15

17

19

21

23

25

27

29

31

Sessions

Perc

en

tag

e o

f In

terv

als

wit

h P

B

0%

10%

20%

30%40%

50%

60%

70%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33

Sessions

Perc

en

tag

e o

f In

terv

als

wit

h P

B

0%

10%

20%

30%

40%

50%

60%

70%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33

Sessions

Perc

en

tag

e o

f In

terv

als

wit

h P

B

0%10%20%30%40%50%60%70%80%

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33

Sessions

Perc

en

tag

e o

f In

terv

als

wit

h P

B

Carl

Angelissa

Marion

Aaron

Aaron Moves to Toledo

Baseline Nifty Intervention

Baseline Nifty Intervention

1. What is

research question?

2. Are Baseline Data Acceptable?

3. Is intervention with Angelisa at Day 11 acceptable?

4. What issues exist at Day 21?

5. Suggest options to save study

USING FUNCTION-BASED ACADEMIC AND BEHAVIOR INTERVENTION TO REDUCE PROBLEM BEHAVIORS AND IMPROVE ENGLISH READING PERFORMANCE FOR ENGLISH LANGUAGE LEARNERS IN ELEMENTARY GRADES IN THAILAND

Chanisa Apichatabutra

Research Question

• Is there a functional relationship between a function-based academic and behavior intervention that (a) is based on behavioral function, (b) employs effective literacy instruction, and (c) is matched to the learners’ culture and a decrease in level of problem behaviors during English reading class for third and fourth grade ELLs in Thailand?

Application• Define research question• Identify design adequacy• Interpret the extent to which the data document a functional

relation.

Figure Baseline 1 Treatment Baseline 2 Treatment 2

Baseline 1 Treatment 1 Baseline 2 Treatment 2

Self-management

DRO

Define research question(s)Define design

What effects are documented?

ies summer research institute: single-case intervention design and analysis day 2 8:45-10:15 rob...

Documents