gender effects in a randomized trial of individual tutoring with children in care robyn marquis...
TRANSCRIPT
Gender effects in a randomized trial of individual tutoring with children in care
Robyn Marquis & Robert J. Flynn
School of Psychology& Centre for Research on Educational and
Community Services (CRECS)University of Ottawa (Canada)
EUSARF 2014, Copenhagen September 3, 2014
Canada & Ontario
Low educational achievement of young people in care:Research in Canada
• In Canada, results similar to those in USA & UK• Flynn & Biro (1998): higher rates of suspension and
grade retention than for peers in general population• Flynn et al. (2004): In samples of young people in
care:– 10-15 years of age: 80% scored in same range
as lowest third of general Canadian population on parental ratings of reading, spelling, and math
– 5-9 years of age: 78% scored in same range of lowest third of Canadian population (same criteria)
• Effect size = size of effect of intervention• Cohen’s d or Hedges g (nearly identical)• Criteria for effect sizes in education:
– What Works Clearinghouse (2011): 0.25– Lipsey et al. (2012) (medians):
• Individual interventions: 0.29• Small-group interventions: 0.22• Classroom: 0.08• Whole school: 0.14• Overall: 0.18
A note on effect sizes in education
Tutoring: A useful intervention• Systematic review & meta-analysis by
Ritter et al. (2006, 2009):– Studies of children in general population– 21 randomized studies, 28 cohorts– Tutoring produced positive effects:
• Reading overall (d = 0.30)*• Reading global (d = 0.26)*• Reading oral fluency (d = 0.30)*• Reading letters & words (d = 0.41)*• Reading comprehension (d = 0.18)• Writing (d = 0.45)*• Mathematics (d = 0.27)
Direct-Instruction Tutoring & Maloney’s Teach Your Children Well
• Direction-instruction teaching method:– Well-organized, structured, effective method
of teaching reading & math skills – For special & general education students – See National Institute for Direct
Instruction web site (http://www.nifdi.org/)• M. Maloney’s Teach Your Children Well:
– DI-based (http://www.maloneymethod.com/)– Combined with behavior management– Uses tutor manuals, learn-to-read series of
books, workbooks, math CD-ROM, training
Our randomized trial(Flynn et al., 2012)
• Collaboration between:– 9 Children’s Aid Societies in Ontario &– University of Ottawa (CRECS)
• Two main questions:1. Does individual direct-instruction tutoring
help children living in foster care to catch up in reading & math?
2. Do girls and boys benefit equally from direct-instruction tutoring?
Method
• Participants: 77 foster children– Children in foster care (grades 2-7, ages 6-13) and
their foster parents (tutors)– Randomly assigned to control or intervention
groups
• 2008-2009 school year– Wait-list control group (n = 35)– Intervention group (n = 42): Tutoring by foster
parents, using Maloney’s TYCW method, for 30 weeks, 3 hrs/week
• Outcome measure:
– Wide Range Achievement Test (WRAT4): • Word Reading• Sentence Comprehension• Reading Composite• Spelling• Math Computation
– Mental health measures
Method
Method - Analysis Sample
• Foster children reassessed at post-test: – Total N = 64– 30 children who had actually received the tutoring
intervention – 34 children in wait-list control condition
• Intervention and control conditions still equivalent, despite attrition
• Question no. 1: Does individual direct-instruction tutoring help children living in foster care to catch up in reading & math?
Results
WRAT4 Word Reading: Results at post-test (N = 64)
Pre-test Post-test95
97
99
101
103
105
Tutoring (n = 30)
Assessment Occasion
Me
an
Sta
nd
ard
Sco
re
(g = .19, p = .19, 1-tailed, ns;post-test scores adjusted for pre-test scores)
WRAT4 Reading Comprehension:Results at post-test (N=64)
Pre-test Post-test95
97
99
101
103
105
Tutoring (n = 30)
Assessment Occasion
Me
an
Sta
nd
ard
Sco
re
(g = .38, p = .035, 1-tailed;post-test scores adjusted for pre-test scores
WRAT4 Reading Composite:Results at post-test (N = 64)
Pre-test Post-test95
97
99
101
103
105
Tutoring (n = 30)
Assessment Occasion
Me
an
Sta
nd
ard
Sco
re
(g = .29, p = .096, 1-tailed;post-test scores adjusted for pre-test scores
WRAT4 Spelling: Results at post-test (N = 64)
Pre-test Post-test95
97
99
101
103
105
Tutoring (n = 30)
Assessment Occasion
Me
an
Sta
nd
ard
Sco
re
(g = -.08, p = .37, 2-tailed, ns;post-test scores adjusted for pre-test scores)
WRAT4 Math Computation:Results at post-test (N = 64)
Pre-test Post-test83
85
87
89
91
93
95
Tutoring (n = 30)
Assessment Occasion
Me
an
Sta
nd
ard
Sco
re
(g = .46, p = .009, 1-tailed;post-test scores adjusted for pre-test scores)
• Question no. 2:Do girls and boys benefit equally from direct-instruction tutoring?
Results
WRAT4 Word Reading:Pre/post change, by gender & condition
PRE-TEST POST-TEST9092949698
100102104106108110
TUTORING (n = 17) CONTROL (n = 19)
*PRE-TEST POST-TEST
90
94
98
102
106
110
TUTORING (n = 13) CONTROL (n = 15)
*
GIRLS (d =.39) BOYS (d = .01)
(*p < .05, 2-tailed)
WRAT4 Sentence Comprehension:Pre/post change, by gender & condition
PRE-TEST POST-TEST9092949698
100102104106108110
TUTORING (n = 17) CONTROL (n = 19)
*
PRE-TEST POST-TEST9092949698
100102104106108110
TUTORING (n = 13) CONTROL (n = 15)
*
GIRLS (d =.12) BOYS (d = .44)
(*p < .05, 2-tailed)
PRE-TEST POST-TEST9092949698
100102104106108110
TUTORING (n = 13) CONTROL (n = 15)
*
GIRLS (d = .25)
PRE-TEST POST-TEST9092949698
100102104106108110
TUTORING (n = 17) CONTROL (n = 19)
*
BOYS (d = .19)
(*p < .05, 2-tailed)
WRAT4 Reading Composite:Pre/post change, by gender & condition
WRAT4 Spelling:Pre/post change, by gender & condition
GIRLS (d = .15)
PRE-TEST POST-TEST9092949698
100102104106108110
TUTORING (n = 17) CONTROL (n = 19)
*
BOYS (d = .19)
PRE-TEST POST-TEST9092949698
100102104106108110
TUTORING (n = 13) CONTROL (n = 15)
*
(*p < .10, 2-tailed)
WRAT4 Math Computation:Pre/post change, by gender & condition
GIRLS (d = .41)
PRE-TEST POST-TEST80828486889092949698
100
TUTORING (n = 17) CONTROL (n = 19)
*
BOYS (d = .21)
PRE-TEST POST-TEST80828486889092949698
100
TUTORING (n = 13) CONTROL (n = 15)
*
(*p < .05, 2-tailed)
Results – Conclusionsregarding gender effects
Girls:– Made statistically significant gains on 4 out of 5
WRAT4 outcome measures– d > median of .29 on Word Reading and Math
ComputationBoys:
– Made statistically significant gains on 3 out of 5 WRAT4 outcome measures
– d > median of .29 on Sentence Comprehension
Overall conclusions • Tutoring by foster parents helps foster
children to catch up in reading and math• Girls and boys both benefit in reading and
math • More well-controlled evaluations of
interventions are needed
Thank you for your attention
• References: For papers by Forsman &
Vinnerljung (2012), Flynn et al. (2012), and Harper & Schmidt (2012), see special issue of Children and Youth Services Review, 34 (6), June, 2012, on improving educational outcomes of young people in care.
• Contact: Robert Flynn ([email protected]). Feel free to write to me by e-mail