what works regarding social skills interventions using single subject design jeffrey chenier, m.a.,...
TRANSCRIPT
![Page 1: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/1.jpg)
What Works Regarding Social Skills Interventions Using Single Subject
Design
Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa Libster, M.A., Kristen O’Leary, Haley York, Natalie Robichaux
and Frank Gresham, Ph.D.
![Page 2: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/2.jpg)
Introduction
• Scientifically Based Research – Section 9101(37) NCLB:
Research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs
(No Child Left Behind Act, 2001)
![Page 3: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/3.jpg)
Introduction
• What Works Clearinghouse (2006): Group Design– Meet Evidence Standards:
• “well designed and implemented randomized controlled trials”
– Meet Evidence Standards with reservations: • “quasi-experiments with equating and no severe
design or implementation problems or randomized clinical trials with severe design or implementation problems”
![Page 4: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/4.jpg)
Introduction
• What Works Clearinghouse (2010): Single Case (SC)– Meet Evidence Standards
• IV must be systematically manipulated, with the researcher determining when and how the IV conditions change
• Each outcome variable must be measured systematically over time by more than one assessor, and the study needs to have IOA calculated 20% of the time in each condition, and IOA percentage must meet minimum thresholds
– 0.80 IOA or 0.60 Cohen’s Kappa• Study must include at least three attempts to demonstrate an
intervention effect at three different points in time or with three different phase repetitions
– Phase must have a minimum of three data points
– Effect size estimation follows if a study has either Strong Evidence or Moderate Evidence
![Page 5: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/5.jpg)
Introduction
• Meta-Analysis– Strube & Hartmann (1982)
• Objective method for summarizing a body of empirical findings
– Emphasizes the direction and magnitude of effects across studies for a particular intervention
![Page 6: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/6.jpg)
What Works Clearinghouse (2010)• No agreed upon method or gold
standard to calculate effect sizes from single-case design research – Problems
• How to quantify the effect?– How accurate is the effect?
• How comparable are the effects across other SC designs?
• How comparable are the effects compared to group design effect sizes?
![Page 7: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/7.jpg)
Current Effect Size Estimators (WWC, 2010)• Nonparametric Methods
– Percentage of Nonoverlapping Data (PND), Percentage of All Nonoverlapping Data (PAND), Percent Exceeding the Median (PEM)
• “Distributional properties of these measures are unknown, so standard errors and statistical tests are not formally justified.”
– Additionally, trend is not addressed • Because of the lack of statistical justification, only use if an
approximate size of the effect is desired.• Wolery et al. (2010) compared four overlapping methods to
visual inspection of effect and each method had its own host of issues, so much that they called for their abandonment
– Visual analysis only agreed 121/160 on whether the treatment was effective or not
![Page 8: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/8.jpg)
Current Effect Size Estimators (WWC, 2010)
• Parametric Methods– Regression Estimates
• Advantages– Familiarity– Ability to model trends– Ability to attain an Effect Size from a single case
• Disadvantages– Inability to deal with complex structures present in
single case design
![Page 9: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/9.jpg)
Current Effect Size Estimators (WWC, 2010)
• Parametric Methods– Multilevel Modeling
• Advantages– Ability to account for complexity of design
• Disadvantages– Unfamiliarity– Technically challenging and time consuming– Different metric from group design Effect Sizes,
therefore the estimate is not comparable
![Page 10: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/10.jpg)
Current Effect Size Estimators (WWC, 2010)• Quantitative Methods
– Differing methods to calculate a Standardized Mean Difference statistic (current study)
• Advantages– Encourages inclusion of SC designs in evaluating effects of
interventions– Potentially gives another method in which to rank order
interventions• Disadvantages
– Not completely comparable to group design research» Pooled within-group variance not comparable to pooled
within phase variance– Small n leads to imprecise estimates– Trend is not assessed
![Page 11: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/11.jpg)
Summary of Effect Size Estimators for SC Design (WWC, 2010)
• Simply put, science is not there yet• Nonparametric estimators should be reported
with a parametric estimator (regression)– Multilevel methods are not ready
• Quantitative methods are not as statistically sound as they should be, but the base from which to build is present
![Page 12: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/12.jpg)
Social Skills
• Learned behaviors that enable positive interactions and allow for escape/avoidance of negative interactions
• Academic Enablers (DiPerna & Elliott, 2002)– Better predictor of academic achievement in 8th
grade than 3rd grade academic achievement (Caprara et. al, 2000)
• Myriad of problems co-occurring with social skills deficits– Both externalizing and internalizing
![Page 13: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/13.jpg)
• Does social skills training work?– Gresham, Cook, Crews, and Kern, 2004
Meta-Analysis n studies
ES g ES rBESD Control
BESD Treatment
Ang & Hughes (2001) 38 .62 .30 35% 65%
Beelman et al. (1994) 49 .47 .23 38% 62%
Losel & Beelmn (2003) 84 .38 .19 40% 60%
Schneider (1992) 79 .89 .40 30% 70%
Schneider & Byrne (1985) 51 .65 .31 34% 66%
Quinn et al. (1999) 35 .20 .10 45% 55%
Means M = .60 M = .29 M = 35% M = 65%
Introduction
![Page 14: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/14.jpg)
Introduction
![Page 15: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/15.jpg)
Introduction
• Godbold et. al, 2010– Contrast Analyses
![Page 16: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/16.jpg)
Introduction
• Research question– Evidence is there for Primary Programs; is
there evidence for Secondary Programs?
![Page 17: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/17.jpg)
Other Meta Analyses Since 2000Meta Analysis Disability Type n studies Intervention Type ES statistic ES Degree of Effect
Bellini & Akullian, 2007 Autism Spectrum 15 Video Modeling PND 81% Effective
Bellini & Akullian, 2008 Autism Spectrum 7 Video Self Monitoring PND 77% Questionable/Effective
Bellini et al., 2007 Autism 15 Child Specific PND 71% Questionable
Bellini et al., 2007 Autism 7 Collateral Skills PND 75% Questionable
Bellini et al., 2007 Autism 20 Comprehensive PND 72% Questionable
Bellini et al., 2007 Autism 10 Peer Mediated PND 62% Questionable
Bellini et al., 2007 Autism 55 Total PND 70% Questionable
Kokina & Kern, 2010 Autism Spectrum 18 Social Stories PND 60% Questionable
Wang & Spillane, 2009 Autism 2 CBT d .47-1.24, .24-.59 Medium - Large, Small - Medium
Wang & Spillane, 2009 Autism 1 CBT PND 100% Very Promising
Wang & Spillane, 2009 Autism 9 Others PND 80.77% Effective
Wang & Spillane, 2009 Autism 9 Peer Mediated PND 60.69% Questionable
Wang & Spillane, 2009 Autism 6 Social Stories PND 67.21% Questionable
Wang & Spillane, 2009 Autism 11 Video Modeling PND 84.25% Effective
Durlak & Weissberg, 2010 Typical 68
After School Programs (Targeting
Positive Social Behaviors)
g 0.19 Small (Significantly different from zero)
Schneider et al, 2008 Autism 19 Social Behavior phi 0.72 Large
![Page 18: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/18.jpg)
Summary
• 7 total studies– 6 with Autism Spectrum, 1 with Typically
Developing
• Multiple interventions available• Effectiveness
– 3 very effective– 3 moderately effective– 1 not as effective (but still statistically significant)
![Page 19: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/19.jpg)
Method
• Literature Search, 2000-2009– Keyword Search in PsycINFO
– 5940 Articles Total
Social Skills +
Competence
Intervention
Training
![Page 20: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/20.jpg)
Method
• Coding 1 Primary Inclusionary Criteria– No Books, Reviews, Meta-Analyses, Group Designs,
Dissertations• Coding 1 Secondary Inclusionary Criteria
– “Is study a social skills intervention or does it target a social skill?” (YES)
– Does study focus on ages 3-21, or through high school? (YES)
– Does study target drugs, alcohol, or sexual offenders? (NO)
• 296 studies remained, 100% IOA in coding 1 (approximately 22% of articles)
![Page 21: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/21.jpg)
Method• Coding 2 Primary Inclusionary Criteria
– Is the full article in English?– Does article include single subject graphs?
• No AB design
• Coding 2 Secondary Inclusionary Criteria– Does study fit our Social Skills definition?
• Gresham, Van, and Cook, 2006:– Facilitates initiating and maintain positive social relationships– Contributes to peer acceptance and friendship development– Results in satisfactory school adjustment– Allows individuals to cope with and adapt to the demands of the social
environment
– Is the study not part of a larger treatment package?• Coded 190 unique studies
– 64 studies on to Coding 3 (IOA = 92% for 38% of studies)
![Page 22: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/22.jpg)
Method
• Coding 3 Primary Inclusion Criteria– If one participant, more than 1 replication
across setting or behavior– Presence of variability in baseline and
treatment conditions across at least 2 participants, settings, or behaviors
– Graphs in which UnGraph was able to score – 40 studies eligible for analysis (IOA 100% on
30% of studies)
![Page 23: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/23.jpg)
Method
– Design Type and Subtype
– Research Question– Main Unit of
Comparison– Participant Info– Phase Info– Dataset Info
• Measurement Strategy• DV information• IOA• Treatment Efficacy
– Study Quality• Three replications
across or within?• Treatment Integrity?• IV Operationally
Defined?• DV Operationally
Defined?• IOA
• Coding 3
![Page 24: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/24.jpg)
Method
• Data Extraction– UnGraph (Biosoft, 2004)
• Extracts numerical data from graphs and puts it into Microsoft Excel
– High reliability and validity in collecting data from single subject graphs (Shadish et al., 2009)
![Page 25: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/25.jpg)
Method
• Effect Size Calculation (Shadish, 2007)– G = (Mt – Mb) / sp
• yields a standardized mean difference statistic
– Currently the best quantitative method available, but not absolutely accurate
![Page 26: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/26.jpg)
Method1) Calculate Mean of Baseline and Tx Panels One, Two, and Three
2) Calculate BL and Tx Mean of Means across the three panels
3) Calculate Standard Deviation of BL and Tx from the Mean of Means
4) Calculate Effect Size for Positive Social Interactions
![Page 27: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/27.jpg)
Results
Participants # Disability Type # Ethnicity #
Total # 148 ADHD 24 % Reported of n 41.10%
Age 2-5 28 ODD 9 Caucasian 28
Age 6-8 8 Asperger's Disorder 9 African American 21
Age 9-12 50 Autism 37 Hispanic 6
Age 13-18 18 BD, EBD, ED 19 Asian 1
Unspecified 8 Language Delay 2 Bi-Racial 1
Male 123 Intellectual Disability 9 Vietnamese 1
Female 23 Typically Developing 44 Latino 1
Classrooms 2 PDD 1 Native American 1
Williams 1
![Page 28: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/28.jpg)
JournalsJournal Name # of Articles
Education & Treatment of Children 5Behavioral Disorders 5Focus on Autism and Other Developmental Disabilities 5Journal of Positive Behavior Interventions 4Psychology in the Schools 3Behavior Modification 2Therapeutic Recreation Journal 2Journal of Autism and Developmental Disorders 2Education and Training 2Journal of Applied Behavior Analysis 2Behavior Therapy 1School Psychology Quarterly 1Remedial and Special Education 1Topics in Early Childhood Special Education 1Journal of Early Intervention 1Journal of the Association for Persons with Severe Handicaps 1International Journal of Play Therapy 1Education Psychology 1
Educational Psychology in Practice 1
![Page 29: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/29.jpg)
Results
• Are social skills interventions evaluated by single subject methods effective?
• Yes• g=3.06
![Page 30: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/30.jpg)
Results
• Independent Variable
IV n ESDirect Instruction 17 2.79Computer-Based 3 2.02*Peer Mediated 5 2.67
Self Management 6 3.53Social Stories 4 3.94
SODA 2 5.54*Reinforcement Based 3 2.17
*p<.05
![Page 31: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/31.jpg)
Results
• Independent Variable– Largest effects occurred with interventions
that target self-awareness / self-monitoring of behavior as the primary independent variable
![Page 32: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/32.jpg)
Results
• Dependent VariableDV n ES
AET/On Task Bx 5 1.57Total Destructive Behaviors (TBD)/ Problem
Behaviors / Negativistic Bx 7 1.30Prosocial Behaviors
Cooperative Play 2 5.54*Positive Statements, Terminations 4 2.12
Prosocial Behavior 13 2.80Sportsmanship 3 2.01
Social Play Skills - Recess 3 3.18SILA 2 1.21*
Seeking Help when lost 1 6.92*
*p<.05
![Page 33: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/33.jpg)
Results
• Dependent Variable– Larger effects are seen when targeting
prosocial behavior over problem behavior
![Page 34: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/34.jpg)
Results
Recipient Director
Recipient n ES
Student 37 3.15*
Classroom 1 2.68
Teacher 1 2.7
Peer 1 2.34
Director n ES
Experimenter 14 3.41
Teacher 10 3.02
Student 4 2.48
Computer 3 2.02*
Coach/Caregiver 3 2.04*
Interventionist 4 3.62*
Peer 2 3.81*
*p<.05 *p<.05
![Page 35: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/35.jpg)
Results
• Recipient and Director– Largest effects are seen when the student
is the target of the intervention and when either an independent experimenter/interventionist or peer is the director of the intervention
![Page 36: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/36.jpg)
Results
• Disability
Disability n ESAutism 13 4.04*
Asperger's 3 4.00*ADHD/ODD 4 2.47ED/BD/EBD 4 2.31
Language Impaired 1 2.68Intellectual Disability 2 1.82*Typically Developing 13 2.48
*p<.05
![Page 37: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/37.jpg)
Results
• Disability*– Interventions are most effective with
children who have disabilities along the Autism spectrum
![Page 38: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/38.jpg)
Results
• Setting
Setting n ES
School / Classroom 23 3.29*
School settings not classroom
6 2.70
Recess 5 2.43*
College Campus 2 2.60
Other 4 3.25*
![Page 39: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/39.jpg)
Results
• Setting– The largest effects were seen when
interventions were implemented in schools
![Page 40: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/40.jpg)
Results
• Number of Components
# Components n ES
2 4 4.02*
3 9 2.66
4 8 3.38*
5 3 2.31
6 10 3.02
7 4 3.34
8 1 1.84*
9 1 2.68
![Page 41: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/41.jpg)
Results
• Components**– Increasing the number of components in
an intervention did not increase the magnitude of effect
• Size may not matter, quality matters
![Page 42: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/42.jpg)
Results
• Treatment Integrity
Integrity n ES
Monitored 3 2.55
No 19 2.71
Yes 18 3.46*
![Page 43: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/43.jpg)
Results
• Treatment Integrity*– Studies that report percentage of integrity
had largest effects
![Page 44: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/44.jpg)
Results
• Reinforcement
Reinforcement Provided n ES
Yes 23 2.88
No 17 3.29
![Page 45: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/45.jpg)
Results• Reinforcement**
– No difference in these studies in regards to reinforcement
• Could be a definitional issue. Coded articles that specified reinforcement given, not necessarily lack of reinforcement
– Reinforcement should be provided if necessary to acquire behavior change
• Quality of intervention may be more important for some
![Page 46: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/46.jpg)
Results
• Study Quality
Quality n ES
3-3.5 5 3.49
4-4.5 22 2.30
5 13 4.16
![Page 47: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/47.jpg)
Results
• Study Quality*– Highest effects were found in studies that
had highest quality ratings, although not statistically significant
– Studies/interventions should be implemented with highest quality possible
![Page 48: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/48.jpg)
Summary of ResultsContrasting Significantly Higher Significantly Lower
IV Self Awareness(1) Computer-Based IVsDV Prosocial Behavior Problem Behavior
Recipient Student
Director Peer, Interventionist Computer, Coach/CaregiverDisability(2) Autism Spectrum Intellectual Disability
Setting School RecessComponents(3) Less More
Integrity Taken(2) Yes NoReinforcement(3) Neither NeitherStudy Quality Higher(4) Lower
1) Mixture of Ivs, all having self awareness qualities2) Matches effect of group design study3) Opposite of effect of group design study4) Not significantly higher, but higher, and similar to our group design finding
![Page 49: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/49.jpg)
Limitations
• Effect size estimator not entirely accurate
• No correction for small sample size
• Stringent selection criteria / data analytic method responsible for abandonment of nearly 40% of single subject studies– Both parametric and nonparametric methods
would have had a larger n
![Page 50: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/50.jpg)
Discussion and Future Directions• All interventions were effective
– If assessment leads to a social skills deficit, with almost any kind of student, there are interventions that work
• Single subject meta-analyses are not yet as informative/definitive as they could be, but current best practice is still to aggregate magnitude and direction of effects across studies– Calculate results using other available methods
and comparing effects (NASP in Philly 2012????)
![Page 51: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/51.jpg)
Questions and Comments?
• For additional copies of this presentation, either check the NASP website or contact Jeffrey Chenier at [email protected]
![Page 52: What Works Regarding Social Skills Interventions Using Single Subject Design Jeffrey Chenier, M.A., Aaron J. Fischer, Katherine Hunter, Emily Patty, Lisa](https://reader035.vdocument.in/reader035/viewer/2022062618/5514b843550346f06e8b6593/html5/thumbnails/52.jpg)
Selected Citations
• Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M & Shadish, W. R. (2010). Single case designs technical documentation. Retrieved from What Works Clearinghouse website: http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.
• Wolery, M., Busick, M., Reichow, B., & Barton, E.E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44, 18-28.