Gender effects in a randomized trial of individual tutoring with children in care

Robyn Marquis & Robert J. Flynn

School of Psychology& Centre for Research on Educational and

Community Services (CRECS)University of Ottawa (Canada)

EUSARF 2014, Copenhagen September 3, 2014

Canada & Ontario

Low educational achievement of young people in care:Research in Canada

• In Canada, results similar to those in USA & UK• Flynn & Biro (1998): higher rates of suspension and

grade retention than for peers in general population• Flynn et al. (2004): In samples of young people in

care:– 10-15 years of age: 80% scored in same range

as lowest third of general Canadian population on parental ratings of reading, spelling, and math

– 5-9 years of age: 78% scored in same range of lowest third of Canadian population (same criteria)

• Effect size = size of effect of intervention• Cohen’s d or Hedges g (nearly identical)• Criteria for effect sizes in education:

– What Works Clearinghouse (2011): 0.25– Lipsey et al. (2012) (medians):

• Individual interventions: 0.29• Small-group interventions: 0.22• Classroom: 0.08• Whole school: 0.14• Overall: 0.18

A note on effect sizes in education

Tutoring: A useful intervention• Systematic review & meta-analysis by

Ritter et al. (2006, 2009):– Studies of children in general population– 21 randomized studies, 28 cohorts– Tutoring produced positive effects:

• Reading overall (d = 0.30)*• Reading global (d = 0.26)*• Reading oral fluency (d = 0.30)*• Reading letters & words (d = 0.41)*• Reading comprehension (d = 0.18)• Writing (d = 0.45)*• Mathematics (d = 0.27)

Direct-Instruction Tutoring & Maloney’s Teach Your Children Well

• Direction-instruction teaching method:– Well-organized, structured, effective method

of teaching reading & math skills – For special & general education students – See National Institute for Direct

Instruction web site (http://www.nifdi.org/)• M. Maloney’s Teach Your Children Well:

– DI-based (http://www.maloneymethod.com/)– Combined with behavior management– Uses tutor manuals, learn-to-read series of

books, workbooks, math CD-ROM, training

Our randomized trial(Flynn et al., 2012)

• Collaboration between:– 9 Children’s Aid Societies in Ontario &– University of Ottawa (CRECS)

• Two main questions:1. Does individual direct-instruction tutoring

help children living in foster care to catch up in reading & math?

2. Do girls and boys benefit equally from direct-instruction tutoring?

Method

• Participants: 77 foster children– Children in foster care (grades 2-7, ages 6-13) and

their foster parents (tutors)– Randomly assigned to control or intervention

groups

• 2008-2009 school year– Wait-list control group (n = 35)– Intervention group (n = 42): Tutoring by foster

parents, using Maloney’s TYCW method, for 30 weeks, 3 hrs/week

• Outcome measure:

– Wide Range Achievement Test (WRAT4): • Word Reading• Sentence Comprehension• Reading Composite• Spelling• Math Computation

– Mental health measures

Method

Method - Analysis Sample

• Foster children reassessed at post-test: – Total N = 64– 30 children who had actually received the tutoring

intervention – 34 children in wait-list control condition

• Intervention and control conditions still equivalent, despite attrition

• Question no. 1: Does individual direct-instruction tutoring help children living in foster care to catch up in reading & math?

Results

WRAT4 Word Reading: Results at post-test (N = 64)

Pre-test Post-test95

97

99

101

103

105

Tutoring (n = 30)

Assessment Occasion

Me

an

Sta

nd

ard

Sco

re

(g = .19, p = .19, 1-tailed, ns;post-test scores adjusted for pre-test scores)

WRAT4 Reading Comprehension:Results at post-test (N=64)

Pre-test Post-test95

97

99

101

103

105

Tutoring (n = 30)

Assessment Occasion

Me

an

Sta

nd

ard

Sco

re

(g = .38, p = .035, 1-tailed;post-test scores adjusted for pre-test scores

WRAT4 Reading Composite:Results at post-test (N = 64)

Pre-test Post-test95

97

99

101

103

105

Tutoring (n = 30)

Assessment Occasion

Me

an

Sta

nd

ard

Sco

re

(g = .29, p = .096, 1-tailed;post-test scores adjusted for pre-test scores

WRAT4 Spelling: Results at post-test (N = 64)

Pre-test Post-test95

97

99

101

103

105

Tutoring (n = 30)

Assessment Occasion

Me

an

Sta

nd

ard

Sco

re

(g = -.08, p = .37, 2-tailed, ns;post-test scores adjusted for pre-test scores)

WRAT4 Math Computation:Results at post-test (N = 64)

Pre-test Post-test83

85

87

89

91

93

95

Tutoring (n = 30)

Assessment Occasion

Me

an

Sta

nd

ard

Sco

re

(g = .46, p = .009, 1-tailed;post-test scores adjusted for pre-test scores)

• Question no. 2:Do girls and boys benefit equally from direct-instruction tutoring?

Results

WRAT4 Word Reading:Pre/post change, by gender & condition

PRE-TEST POST-TEST9092949698

100102104106108110

TUTORING (n = 17) CONTROL (n = 19)

*PRE-TEST POST-TEST

90

94

98

102

106

110

TUTORING (n = 13) CONTROL (n = 15)

*

GIRLS (d =.39) BOYS (d = .01)

(*p < .05, 2-tailed)

WRAT4 Sentence Comprehension:Pre/post change, by gender & condition

PRE-TEST POST-TEST9092949698

100102104106108110

TUTORING (n = 17) CONTROL (n = 19)

*

PRE-TEST POST-TEST9092949698

100102104106108110

TUTORING (n = 13) CONTROL (n = 15)

*

GIRLS (d =.12) BOYS (d = .44)

(*p < .05, 2-tailed)

PRE-TEST POST-TEST9092949698

100102104106108110

TUTORING (n = 13) CONTROL (n = 15)

*

GIRLS (d = .25)

PRE-TEST POST-TEST9092949698

100102104106108110

TUTORING (n = 17) CONTROL (n = 19)

*

BOYS (d = .19)

(*p < .05, 2-tailed)

WRAT4 Reading Composite:Pre/post change, by gender & condition

WRAT4 Spelling:Pre/post change, by gender & condition

GIRLS (d = .15)

PRE-TEST POST-TEST9092949698

100102104106108110

TUTORING (n = 17) CONTROL (n = 19)

*

BOYS (d = .19)

PRE-TEST POST-TEST9092949698

100102104106108110

TUTORING (n = 13) CONTROL (n = 15)

*

(*p < .10, 2-tailed)

WRAT4 Math Computation:Pre/post change, by gender & condition

GIRLS (d = .41)

PRE-TEST POST-TEST80828486889092949698

100

TUTORING (n = 17) CONTROL (n = 19)

*

BOYS (d = .21)

PRE-TEST POST-TEST80828486889092949698

100

TUTORING (n = 13) CONTROL (n = 15)

*

(*p < .05, 2-tailed)

Results – Conclusionsregarding gender effects

Girls:– Made statistically significant gains on 4 out of 5

WRAT4 outcome measures– d > median of .29 on Word Reading and Math

ComputationBoys:

– Made statistically significant gains on 3 out of 5 WRAT4 outcome measures

– d > median of .29 on Sentence Comprehension

Overall conclusions • Tutoring by foster parents helps foster

children to catch up in reading and math• Girls and boys both benefit in reading and

math • More well-controlled evaluations of

interventions are needed

Thank you for your attention

• References: For papers by Forsman &

Vinnerljung (2012), Flynn et al. (2012), and Harper & Schmidt (2012), see special issue of Children and Youth Services Review, 34 (6), June, 2012, on improving educational outcomes of young people in care.

• Contact: Robert Flynn (rflynn@uottawa.ca). Feel free to write to me by e-mail