non-overlap methods in single case research methodology erin e. barton, phd, bcba-d
TRANSCRIPT
![Page 1: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/1.jpg)
Non-Overlap Methods in Single Case Research Methodology
Erin E. Barton, PhD, BCBA-D
![Page 2: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/2.jpg)
Non-overlap Methods
1. PND
2. PEM (ECL)
3. PEM-T
4. PAND
5. R-IRD
6. NAP
7. Tau-U
![Page 3: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/3.jpg)
Rationale: Non-overlap Methods
1. Need to aggregate across studies to determine evidence for practice
Meta-analysis is a well established practice for group experiments
– Magnitude – Aggregate findings – Moderator analyses
![Page 4: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/4.jpg)
Rationale: Non-overlap Methods
1. Need to aggregate across studies to determine evidence for practice
– Many have argued if SCRD will not be included in reviews of evidence-based practices unless an effect size estimator is used
– They are often left out of reviews in disciplines outside of special education
![Page 5: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/5.jpg)
Rationale: Non-overlap Methods
2. Emphasis on “effect sizes” in education research to quantify the magnitude
– Standardized effect sizes are particularly valued– Reviewers compare results across studies having
different outcome measures that otherwise could not be easily compared
![Page 6: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/6.jpg)
Rationale: Non-overlap Methods
3. Meta-analytic techniques are compromised when data are serially dependent – On a single individual– Using the same definitions– Using the same data collection procedures– In the same context – Under the same procedures– Often with short intervals between observations
![Page 7: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/7.jpg)
Rationale: Non-overlap Methods
4. Current meta-analytic techniques are inappropriate for aggregating SCR data
The data patterns must shift consistently in the predicted (therapeutic) direction with each change in experimental condition
Replication logic is used to make judgments about functional relations
The design needs an adequate number of replications of the experimental conditions (internal validity)
![Page 8: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/8.jpg)
Rationale: Non-overlap Methods
5. Non-parametric techniques needed
• Short data sets or few data points• Non-normal or unknown distributions• Unknown parameters
![Page 9: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/9.jpg)
PND: Percent of Non-overlapping Data
![Page 10: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/10.jpg)
PND: Percent of Non-overlapping Data
• One of the oldest of the overlap methods (Scruggs, & Mastropieri, 1998; Scruggs, Mastropieri, & Casto, 1987)
• Used extensively• Easily calculated• Does not assume data are independent
![Page 11: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/11.jpg)
Calculating PND
1. Identify the intended change
2. Drawing a straight line from the highest (or lowest) point in Phase A and counting the number of data point in Phase B above the line
3. Quotient = # above the line / total number in Phase B X 100
![Page 12: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/12.jpg)
Interpreting PND
70% is effective, 50% to 70% is questionable effectiveness, and <50% is no observed effect
(Scruggs & Mastropieri, 1998)
![Page 13: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/13.jpg)
![Page 14: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/14.jpg)
![Page 15: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/15.jpg)
Practice: Calculating PNDSchilling (2004), David
In Seat
A1 to B1:
B1 to A2:
A2 to B2:
Engaged
A1 to B1:
B1 to A2:
A2 to B2:
![Page 16: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/16.jpg)
A1 to B1: 22%
B1 to A2: 100%
A2 to B2: 100%
AVERAGE: 74%
Practice: Calculating PNDSchilling (2004), David
In Seat Engaged
A1 to B1: 100%
B1 to A2: 100%
A2 to B2: 100%
AVERAGE: 100%
![Page 17: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/17.jpg)
![Page 18: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/18.jpg)
![Page 19: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/19.jpg)
Practice: Calculating PNDVaughn (2002)
Disruptive Behavior
Arrival: 100%
Mealtime: 75%
Departure: 100%
AVERAGE: 92%
Engaged
Arrival: 100%
Mealtime: 75%
Departure: 100%
AVERAGE: 92%
![Page 20: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/20.jpg)
Outliers
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
1
2
3
4
5
6
7
8
Days
CB
Rat
e p
er M
inu
te
PND = 0%
PND = 0%
PND = 0%
Functional Relation?
![Page 21: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/21.jpg)
Trends
1 2 3 4 5 6 7 8 9 101112131415161718192021222324252627280123456789
10
Days
CB
Rat
e pe
r M
inut
e
PND = 0%
PND = 0%
PND = 0%
Functional Relation?
![Page 22: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/22.jpg)
Magnitude
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
2
4
6
8
10
12
Days
CB
Rat
e p
er M
inu
te
PND = 100%
PND = 100%
PND = 100%
Functional Relation?
![Page 23: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/23.jpg)
Outliers
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
1
2
3
4
5
6
7
8
Days
CB
Rat
e p
er M
inu
te
PND = 0%
PND = 0%
PND = 0%
Functional Relation?
![Page 24: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/24.jpg)
BUT NO ONE USES A-B-A-B…..
You might be thinking….
![Page 25: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/25.jpg)
0
2
4
6
8
10
12
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
2
4
6
8
10
12
Days
Fu
nct
ion
al R
elat
ion
?
PND = 68%
PND = 43%
PND = 86%Soc
ial I
nitia
tions
to
Pee
rs
![Page 26: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/26.jpg)
PND = 68%
PND = 43%
PND = 86%
PND = 66%
TREND?
![Page 27: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/27.jpg)
0
2
4
6
8
10
12
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
2
4
6
8
10
12
Days
Fu
nct
ion
al R
elat
ion
?
PND = 0%
PND = 0%
PND = 0%Soc
ial I
nitia
tions
to
Pee
rs
![Page 28: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/28.jpg)
PND = 0%
PND = 0%
PND = 0%
PND = 0%
MAGNITUDE?
![Page 29: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/29.jpg)
0
1
2
3
4
5
6
7
8
0
1
2
3
4
5
6
7
8
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28012345678
Days
Fu
nct
ion
al R
elat
ion
?
PND = 100%
PND = 100%
PND = 100%
Soc
ial I
nitia
tions
to
Pee
rs
![Page 30: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/30.jpg)
WHAT DOES THAT MEAN?
You SHOULD be thinking….
![Page 31: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/31.jpg)
PND Flaws
• Compared with consensus visual analysis, PND resulted in an error in about 1 of 5 condition changes—high rate of errors (Wolery, Busick, Reichow, & Barton, 2010)
• Compromised by:– Longer data sets, # of
data points – Variability – Outliers – Trends
Should not be used (Brossart et al., 2013; Kratochwill et al., 2010; Parker & Vannest, 2009)
![Page 32: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/32.jpg)
PND Flaws
Is it an Effect Size?
Replication
Magnitude
Can it supplement visual analysis?
1. Level
2. Trend
3. Variability
4. Immediacy
5. Overlap
6. Consistency Vertical analysis
![Page 33: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/33.jpg)
PAND: Percent of All Non-overlapping Data
![Page 34: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/34.jpg)
PAND: Percent of All Non-overlapping Data
• Not compromised by serial dependency or other data assumptions
• Percentage of data remaining after determining the fewest data points that must be removed to eliminate all between-phase overlap
![Page 35: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/35.jpg)
Calculating PAND
1. Count the total number of data points in comparison
2. Identify how many need data points need to be removed to eliminate overlap
3. Count the number of remaining data points
4. Divide count in step 3 by count in step 1
5. X by 100
![Page 36: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/36.jpg)
Practice: Calculating PANDSchilling (2004), David
In Seat
A1 to B1: 94%
B1 to A2: 100%
A2 to B2: 100%
AVERAGE: 98%
(PND Average was 74%)
Engaged
A1 to B1: 100%
B1 to A2: 100%
A2 to B2: 100%
AVERAGE: 100%
(PND Average was 100%)
![Page 37: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/37.jpg)
Practice: Calculating PANDVaughn (2002)
Disruptive Behavior
Arrival: 100%
Mealtime: 85%
Departure: 100%
AVERAGE: 95%
PND was 92%
Engaged
Arrival: 100%
Mealtime: 85%
Departure: 100%
AVERAGE: 95%
PND was 92%Adding more data points
Extinction burst Variability
![Page 38: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/38.jpg)
0
2
4
6
8
10
12
0
2
4
6
8
10
12
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
2
4
6
8
10
12
Days
PAND = 96%
PAND = 93%
PAND = 93%Soc
ial I
nitia
tions
to
Pee
rs
![Page 39: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/39.jpg)
PAND = 96%
PAND = 93%
PAND = 93%
PAND = 94%
MAGNITUDE?
![Page 40: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/40.jpg)
0
1
2
3
4
5
6
7
8
0
1
2
3
4
5
6
7
8
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28012345678
Days
PAND = 100%
PAND = 100%
PAND = 100%
Soc
ial I
nitia
tions
to
Pee
rs
![Page 41: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/41.jpg)
PAND Flaws
Is it an Effect Size?
Replication
Magnitude
Can it supplement visual analysis?
1. Level
2. Trend
3. Variability
4. Immediacy
5. Overlap
6. Consistency Vertical analysis
Should not be used (Brossart et al., 2013; Manolov et al., 2010)
![Page 42: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/42.jpg)
PEM: Percent Exceeding the Median (Ma, 2006)
![Page 43: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/43.jpg)
PEM: Percent Exceeding the Median (Ma, 2006)
• Not compromised by serial dependency and other data assumptions
• Designed to eliminate problem with baseline datum point being at floor or ceiling
– Designed to not rely on the most extreme datum point
– Less influenced by variability in baseline
![Page 44: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/44.jpg)
Calculating PEM
1. Drawing a line at the median of Phase A data through Phase B data
2. Count the number of data points in Phase B above (or below) the line and divide by the total number of data points in Phase B
![Page 45: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/45.jpg)
PEM Flaws
• Compared with consensus visual analysis, PEM resulted in an error in about 1 of 6 condition changes—high rate of errors (Wolery et al., 2010)
Should not be used (Parker, Vannest, & Davis, 2011)
![Page 46: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/46.jpg)
PEM Flaws
Is it an Effect Size?
Replication
Magnitude
Can it supplement visual analysis?
1. Level
2. Immediacy
3. Overlap
4. Consistency Vertical analysis
![Page 47: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/47.jpg)
PEM-T: Percent Exceeding the Median Trend Line
ECL: Extended Celeration Line
![Page 48: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/48.jpg)
PEM-T: Percent Exceeding the Median Trend Line (Wolery et al., 2010)
• Not compromised by serial dependency and other data assumptions
• Designed to eliminate problem with baseline datum point being at floor or ceiling
– Designed to not rely on the most extreme datum point
– Less influenced by variability in baseline– Less influenced by trends in data
![Page 49: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/49.jpg)
Calculating PEM-T
1. Graph data on semi-logarithmic chart
2. Calculate and draw a split middle line of trend estimation for Phase A data and extend it through Phase B
3. Count # of Phase B data points above/below the split middle line of trend estimation
4. Divide count from Step 4 by # data points in Condition 2 and multiply quotient by 100
![Page 50: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/50.jpg)
PEM-T Flaws
• Compared with consensus visual analysis, PEM-T resulted in an error in about 1 of 8 condition changes—high rate of errors (Wolery et al., 2010)
![Page 51: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/51.jpg)
PEM-T Flaws
Is it an Effect Size?
Replication
Magnitude
Can it supplement visual analysis?
1. Level
2. Trend
3. Variability
4. Immediacy
5. Overlap
6. Consistency Vertical analysis
![Page 52: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/52.jpg)
So what, right?
![Page 53: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/53.jpg)
Social Stories for Children with ASD
• 20 studies met design standards with or without reservation, which exceeded the minimum number of five studies set by the WWC as needed to be represented across studies.
• Across 3 research groups
Qi & Barton, under review
![Page 54: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/54.jpg)
Social Stories for Children with ASD
• Using Non-overlap indices: – 28 participants (51%) had a PND score higher
than 70; 41 (75%) had a PEM score higher than 70; 40 (73%) had a PEM-T score higher than 70; and 50 (91%) had a PDO2 score higher than 70.
• Using visual analyses: – only 13 participants were included across the
one study that provided strong evidence and in the six studies that provided moderate evidence.
Qi & Barton, under review
![Page 55: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/55.jpg)
Social Stories for Children with ASD
• Based on visual analysis, social stories interventions were not considered an EBP according to WWC criteria.
• Based on non-overlap indices, social stories interventions were considered an EBP according to WWC criteria.
Qi & Barton, under review
![Page 56: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/56.jpg)
R-IRD: Robust Improvement Rate Difference
![Page 57: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/57.jpg)
R-IRD: Robust Improvement Rate Difference
• Not compromised by serial dependency and other data assumptions
• Not about rate • IRD calculation begins as PAND, but in a second
step converts the results to two improvement rates (IR), for phase A and B respectively.
• The two IR values are finally subtracted to obtain the “Improvement Rate Difference” (IRD)
• R-IRD requires rebalancing (by hand) of a 2 x 2 matrix
![Page 58: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/58.jpg)
R-IRD: Robust Improvement Rate Difference
• The original IRD article recommended that in the first step, data point ‘removal’ “should be balanced across the contrasted phases” (Parker et al., 2009, p. 141) for more robust results.
• A better robust IRD solution was later described and formalized as “Robust IRD” (R-IRD).
• R-IRD requires rebalancing (by hand) of a 2 x 2 matrix
• IRD is interpreted as the difference in the proportion of high or “improved” scores between phases B and A.
![Page 59: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/59.jpg)
R-IRD: Robust Improvement Rate Difference
• The superior robust version of IRD (R-IRD) requires that quadrants be balanced.– Balancing when a large number of data points
are be removed arbitrarily from one side and a few from the other…
– Does not allow bias in removal of data points from A versus B, as some datasets provide two or more equally good removal solutions.
![Page 60: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/60.jpg)
Calculating R-IRD
1. Determine the fewest data points that must be removed to eliminate overlap
2. Balance quadrant W and Z
3. Then balance Y = A – Phase A: W / (W + Y)
4. Then balance X = B – Phase B: X / (X + Z)
5. R - IRD = B – A
![Page 61: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/61.jpg)
Calculating R-IRD
http://www.singlecaseresearch.org/calculators/ird
Flaws: Length of data can impact (Brossart et al., 2013; Manolov et al., 2011)
![Page 62: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/62.jpg)
NAP: Non-overlap of All Pairs
![Page 63: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/63.jpg)
NAP: Non-overlap of All Pairs
• The percentage of data that improve from A to B or operationally, the percentage of all pairwise comparisons from Phase A to B which show improvement or growth (Parker & Vannest, 2009)
![Page 64: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/64.jpg)
Calculating NAP
1. NAP begins with all pairwise comparisons (#Pairs = nA × nB) between phases.
2. Each paired comparison has one of three outcomes: improvement over time (Pos), deterioration (Neg), or no change over time (Tie).
3. NAP is calculated as (Pos + .5 × Tie) / #Pairs.
![Page 65: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/65.jpg)
Practice: Calculating NAP
1 2 3 4 5 6 7 8 9 10 11 12 130
1
2
3
4
5
6
7
8
Phase A: 0 4 3 0 0Phase B: 5 2 3 5 3 5 6 7
![Page 66: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/66.jpg)
1 2 3 4 5 6 7 8 9 10 11 12 13012345678
# of Pairs = 5*8 = 40#Pos = 34, #Neg = 4, #Tie = 2NAP = (#Pos + .5*#Ties)/#PairsNAP = (34 + .5*2)/40NAP = .875
N=5 N=8
![Page 67: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/67.jpg)
NAP Flaws
Is it an Effect Size?
Replication
Magnitude
Can it supplement visual analysis?
1. Level
2. Trend
3. Variability
4. Immediacy
5. Consistency
6. Overlap Vertical analysis
![Page 68: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/68.jpg)
Tau-U
Extension of NAP – but can control for trend.
![Page 69: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/69.jpg)
Tau-U:(Kendall’s Tau + Mann-Whitney U)
• NAP’s major limitation of insensitivity to data trend led to development of a new index that integrates non-overlap and trend: TauU (Parker, Vannest, Davis, & Sauber, 2011).
• Melding KRC and MW-U are transformations of one another and share the same S sampling distribution
• The Tau-U score is not affected by the ceiling effect present in other non-overlap methods, and performs well in the presence of autocorrelation.
• NAP is percent of non-overlapping data, whereas TauU is percent of non-overlapping minus overlapping data.
• Can control for baseline trends
![Page 70: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/70.jpg)
Calculating Tau-U
Simplest Tau (non-overlap only)• Conduct the same pairwise comparisons
(nA × nB = #Pairs) across phases as is NAP, resulting in a Pos, Neg, or Tie for each pair
• The Tau simple non-overlap form (not considering trend) is Tau = (Pos - Neg) / Pairs
• Tau-U can control for baseline trend
![Page 71: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/71.jpg)
Practice: Calculating TauU
1 2 3 4 5 6 7 8 9 10 11 12 130
1
2
3
4
5
6
7
8
Phase A: 0 4 3 0 0Phase B: 5 2 3 5 3 5 6 7
![Page 72: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/72.jpg)
1 2 3 4 5 6 7 8 9 10 11 12 13012345678
# of Pairs = 5*8 = 40#Pos = 34, #Neg = 4, #Tie = 2Tau-U = (#Pos - #Neg)/#PairsTau-U = (34 - 4)/40Tau-U = .75
N=5 N=8
![Page 73: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/73.jpg)
Practice
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
1
2
3
4
5
6
7
8
Days
CB
Rat
e p
er M
inu
te
Tau-U = -.82
Tau-U = .79Tau-U = -.81
78
67
56 pairs, 2+, 6=, 48-
48 pairs, 40+, 6=, 2-
42 pairs, 1+, 6=, 35-
![Page 75: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/75.jpg)
![Page 76: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/76.jpg)
Calculating Tau-USchilling (2004), David
In Seat
A1 to B1: 1 (0.54, 1.46)
B1 to A2: -1 (-1.59, -.41)
A2 to B2: 1 (.36, 1.64)
PAND Average: 98%
(PND Average was 74%)
Engaged
A1 to B1: .83 (.37, 1.29)
B1 to A2: -1 (-1.59, -.41)
A2 to B2: 1 (.36, 1.64)
PAND Average: 100%
(PND Average was 100%)
![Page 77: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/77.jpg)
Calculating Tau-UVaughn (2002)
Disruptive Behavior
Arrival: -1 (-1.708, -0.23)
Mealtime: -.67 (-1.44, .11)
Departure: -1 (-1.78, -.23)
PAND was 95%
PND was 92%
Engaged
Arrival: 1 (.292, 1.71)
Mealtime: .83 (.06, 1.61)
Departure: 1 (.23, 1.76)
PAND was 95%
PND was 92%
![Page 78: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/78.jpg)
0
1
2
3
4
5
6
7
8
0
1
2
3
4
5
6
7
8
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
2
4
6
8
Days
Tau-U = 1.0S
ocia
l Ini
tiatio
ns t
o P
eers
Tau-U = 1.0
Tau-U = 1.0
![Page 79: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/79.jpg)
0
1
2
3
4
5
6
7
8
0
1
2
3
4
5
6
7
8
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 280
2
4
6
8
Days
Tau-U = 1.0S
ocia
l Ini
tiatio
ns t
o P
eers
Tau-U = 1.0
Tau-U = 1.0
![Page 80: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/80.jpg)
Tau-U Flaws
Is it an Effect Size?
Replication
Magnitude
Can it supplement visual analysis?
1. Trend
2. Level
3. Variability
4. Overlap
5. Immediacy
6. Consistency Vertical analysis
Tau-U is the recommended non-
overlap index
![Page 81: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/81.jpg)
Summary
• Complete non-overlap measures offer the most robust option (NAP, Tau-U)– Complete measures equally emphasize all scores– Incomplete measures emphasize particular scores
(e.g., median)
![Page 82: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/82.jpg)
Summary
• Determination of evidence-based practice does not need to involve summary statistic
• Dynamic, flexible nature of SCRD allow for ongoing decision making while maintaining experimental control
• Replication logic can not be ignored
![Page 83: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/83.jpg)
Summary
• Visual Analysis is complex and involves more than overlap– Graphing is important!
• Effect sizes, non-overlap measures should not take the place of visual analysis
![Page 84: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/84.jpg)
Continue to:
1. Determine if the design supports the demonstration of a functional relation and meets design standards
2. Use systematic visual analysis to determine if data support a functional relational
– Not just behavioral change or effect– Report protocol used and perhaps training of VAs– Include predicted data pattern in RQ
3. Consider magnitude and social validity
4. If using an effect size estimator, test and report assumptions
![Page 85: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/85.jpg)
Develop:
Effect sizes that are: • Is consistent with single case research
design logic • Synthesis of SC studies with similar IVs
and DVs• Synthesis rigorous, experimental studies
using SCR and RCTs
![Page 86: Non-Overlap Methods in Single Case Research Methodology Erin E. Barton, PhD, BCBA-D](https://reader037.vdocument.in/reader037/viewer/2022110210/56649e5f5503460f94b58e5f/html5/thumbnails/86.jpg)
ReferencesBroissart, D. F., Vannest, K. J., Davis, J. L., & Patience, M. A. (2014). Incorporating nonoverlap indices with visual analysis for quantifying intervention effectiveness in single-case experimental designs. Neuropsychological Rehabilitation: An International Journal, 24, 464-491.
Ma, H. H. (2006).An alternative method for quantitative synthesis of single-subject research: Percentage of datapoints exceeding the median. Behavior Modification, 30, 598–617.
Parker, R., & Vannest, K. J. (2008). An improved effect size for single case research: Non-overlap of all pairs (NAP). Behavior Therapy, 40, 357-67.
Parker, R. I., Vannest, K. J., & Brown, L. (2009). The improvement rate difference for single case research. Exceptional Children, 75, 135–150.
Parker, R. I., Vannest, K. J., & Davis, J. L. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35, 303-322.
Wolery, M., Busick, M., Reichow, B., & Barton, E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. Journal of Special Education, 44, 18-28.