Children and Youth Services Review 34 (2012) 1176–1182

Contents lists available at SciVerse ScienceDirect

Children and Youth Services Review

j ourna l homepage: www.e lsev ie r .com/ locate /ch i ldyouth

Preliminary effects of a group-based tutoring program for children in long-termfoster care

Julie Harper a,⁎, Fred Schmidt a,b

a Lakehead University, 955 Oliver Road, Thunder Bay, Ontario, Canada P7B 5E1b Children's Centre Thunder Bay, 283 Lisgar St., Thunder Bay, Ontario, Canada P7C 1R7

⁎ Corresponding author. Tel.: +1 807 251 9326; fax:E-mail addresses: jharper@lakeheadu.ca (J. Harper),

(F. Schmidt).

0190-7409/$ – see front matter © 2012 Elsevier Ltd. Alldoi:10.1016/j.childyouth.2012.01.040

a b s t r a c t

a r t i c l e i n f o

Available online 3 February 2012

Keywords:Direct instructionOut-of-home careAcademic achievementTutoring

Background: Children in foster care are frequently behind in educational achievement (Flynn, Ghzal, Legault,Vandermeulen, & Petrick, 2004) and perform below grade level (Trout, Hagaman, Casey, Reid, & Epstein,2008 for a review).Objective: This study evaluated the effectiveness of a direct instruction literacy and math program (“TeachYour Children Well”; TYCW) in a small-group format to educationally disadvantaged foster care children.Method: In the first year of this two-year study, 68 children in long-term foster care, between grades 2 and

8 inclusive, participated in this randomized control trial intervention. One-half of the children were randomlyassigned to the 25-week experimental TYCW condition, while the other children served as waitlist controls.Children were assessed at baseline and post-intervention on reading decoding, spelling, sentence compre-hension, and mathematic skills using an academic measure of functioning, the Wide Range AchievementTest Forth Edition (WRAT4).Results: Based on preliminary year one data, an analysis of covariance (ANCOVA) demonstrated a statisticallysignificant increase in standard scores on reading decoding and spelling skills for those in the tutoring con-dition, but statistical group differences were not obtained for mathematics or sentence comprehension.Meaningful effect size differences, in the small-to-moderate range, were found in favour of the tutoring inter-vention with respect to reading, spelling, and mathematics skill development. Discussion: Implications of thefindings for improving the educational outcomes of foster children are provided.

© 2012 Elsevier Ltd. All rights reserved.

1. Introduction

As foster care is intended to protect children and has the potentialto remediate areas of deficit and poor functioning, it is essential toevaluative whether this can be successfully accomplished within theexisting resources of the child welfare system. One, among several,important areas of functioning in foster care children that requiresfurther study is the viability of correcting and rehabilitating common-ly found deficits in academic functioning in this vulnerable popula-tion. The focus of this paper will be to evaluate the preliminary yearone data regarding the effectiveness of a group-based direct instruc-tion (DI) tutoring program, Teach Your ChildrenWell (TYCW), for chil-dren in long-term foster care. First, however, the academic needs ofchildren in foster care will be described. This will be followed by a de-scription of the common types of learning problems found in childrenand academic instruction programs that have been developed toremediate academic deficits. The evaluation of one such academicintervention program, TYCW, for children in long term foster care

+1 807 343 7734.fschmidt@childrenscentre.ca

rights reserved.

will be further described. As the DI program implemented was group-based and delivered by volunteer students, the importance of each ofthese components as they relate to successful remedial interventionswill be discussed. Finally, given that over 75% of the children referredto the study were Aboriginal Canadian, the importance of culture as itrelates to the effectiveness of tutoring, specifically DI, will be elaboratedand justified.

1.1. Academic deficits of children in foster care

Children and adolescents in foster care are at a disadvantage in ed-ucational achievement relative to their same-age peers in the generalpopulation (Chamberlain, Moreland, & Reid, 1992; Trout, Hagaman,Casey, Reid, & Epstein, 2008). Research findings indicate that any-where from 33% to 67% of foster care children experience poor aca-demic achievement and require remedial assistance (Trout et al.,2008). Trout et al. (2008) conducted a systematic review of the aca-demic functioning of children and adolescents placed in out-of-homecare in the United States, and consistently found that these youth per-formbelow grade level and in the low to low-average range on academ-ic achievement measures. Furthermore, children in foster care alsoexhibit higher rates of learning disabilities, grade retention (Casey

1177J. Harper, F. Schmidt / Children and Youth Services Review 34 (2012) 1176–1182

et al., 2008), and special education placement (Trout et al., 2008), withfewer foster children completing high school (Courtney, Piliavin,Grogan-Kaylor, & Nesmith, 2001).

Flynn et al. have conducted a number of studies examining theproblem of low educational achievement among foster care youth inout-of-care placements in Canada. Flynn and Biro (1998) found thatchildren in foster care reported higher rates of suspensions and grade re-tention relative to same aged peers. More recently, results indicate that80% of youth between the ages of 10 and 15 scored within the samerange as the lowest third of the general Canadian population onmeasuresof reading, spelling and math (Flynn, Ghazal, Legault, Vandermeulen, &Petrick, 2004). For younger children between five and nine years of age,a similar high percentage of foster care children (i.e., 78%), scored in thesame range as those in the lowest third of the general Canadian popula-tion based on subjective parental ratings (Flynn et al., 2004). Researchconducted in the United States has found similar results to those previ-ously reported in Canada (Burley & Halpern, 2001). The evident delaysin academic achievement skills and school success for children in fostercare highlight the importance of using effective remedial programs toameliorate these weaknesses.

1.2. Remedial programs

Different types of remedial programs have been designed to ad-dress educational deficits for children in the areas of reading, writingand math. Those programs that target phonological awareness tendto demonstrate better outcomes than those that target context orwhole word reading approaches (Torgesen, Wagner, & Rashotte, 1997),with this effect particularly apparent for children with existing weakphonological abilities (Felton, 1993; Torgesen, Morgan, & Davis, 1992).Reinforcing the importance of phonological instruction, a recent meta-analytic study found direct instruction (DI) to be one of the top 3 ef-fective models for urban and low-performing schools (Borman, Hewes,Overman, & Brown, 2003). Furthermore, DIwas the only program of thethree that could be implemented outside of school hours, and did notrequire a school-wide curriculum or multisystem approach (Bormanet al., 2003). DI is conceptualized as a complex skill set wherebychildren are taught each component skill needed to achieve a targetbehaviour (Ryder, Burton, & Silberg, 2006), including instruction onphonetic skills. DI was found to be one of the top three interven-tions because it demonstrated strong effectiveness, generalizability,and a positive effect size (d=+0.21) over other interventions(Borman et al., 2003).

The DI model is associated with three basic principles: (a) lan-guage is broken down into components and taught in isolation withno context, (b) instruction is directed by the teacher, and (c) minimalinput is obtained from the student on instructional content (Stahl,Duffy-Hester, & Stahl, 1998). In essence, DI utilizes explicit teachingto promote reading mastery (Shippen, Houchins, Steventon, & Sartor,2005). As well, many programs have complemented DI instructionwith behavioural management strategies. Remedial instruction pro-grams that have combined DI with contingency management strate-gies in the form of a token economy, have been found to be effectiveat improving academic outcomes for children with disabilities, thosejudged to be at risk for school failure, and a general population ofmiddle school students (Dolezal, Weber, Evavold, Wylie, & McLaughlin,2007).

A recent quasi-experimental study investigated the differentialeffects of DI reading programs for 55 seventh grade students per-forming 2 or more years behind in reading and thus considered tobe struggling in school (Shippen et al., 2005). This study found thatafter six weeks of DI, students showed gains in reading efficiency,reading rate, reading accuracy, and reading fluency (Shippen et al.,2005). These results, however, produced moderate effect size forpre-post outcome measures d=.40 (Shippen et al., 2005). The effec-tiveness of the DI programwas also studied in second grade students

across a two year time period with what the authors termed “basalreading programs” (Ashworth, 1999). The basal reading programsencompassed reading programs typically encountered in academicsettings with a particular focus on meaning and whole word recog-nition (Ashworth, 1999). Those who received the DI interventiondisplayed between 5% and13%higher achievement scores in areas of vo-cabulary (13%), Comprehension (8%) and language (5.4%) when com-pared to those who received a meaning reading program (Ashworth,1999). Other studies support the efficacy of DI interventions on readingand achievement across different grades and ethnicities (Dowdell,1996). Dowdell (1996) investigated DI among an entire minoritysample of sixth grade students, and found a statistically significantdifference as the experimental group had a mean increase of 1.06on the reading achievement subscale of the Iowa Test of Basic Skills,with a change of only .45 by the children in the control group. Animportant limitation of this study, however, is that the conditionswere not randomly assigned (Dowdell, 1996).

A meta-analysis of youth with learning disabilities (Borman et al.,2003), and two randomized control trials, one of young high-risk chil-dren (Cole, Dale, Mills, & Jenkins, 1993) and the other of children infoster care (Flynn et al., 2012-this issue), support the efficacy of DIprograms for improving academic achievement for students. Howev-er, as evidenced by the literature, there is a limited focus on the effec-tiveness of DI for children in foster care populations. Moreover, theresearch which has been done tends to focus on an individual deliv-ery format of DI. The DI model, however, can be delivered in eitheran individual or group format. Given the economies of resourcesand potential ability to reach more children within a group format,it is important to not only consider which intervention program isused but also the modality in which it is given.

1.3. Group-based remedial programming

Several meta-analytic studies have been conducted to explorethe effectiveness of tutoring programs as a supplement for class-room teaching. Generally, one-to-one tutoring has been empiricallyvalidated for students considered at risk for school failure or thoseidentified with a learning disability (Elbaum, Vaughn, Hughes, &Moody, 2000). One-to-one tutoring has been provided by a varietyof individuals including teachers, volunteers, and college students(Elbaum et al., 2000). However, very few studies exist that directlycompare one-to-one tutoring programs with potentially more costeffective small group interventions.

The meta-analytic study of Elbaum et al. (2000) found two studiesin the literature between 1975 and 1998 that compared one-to-oneinterventions with small-group interventions. Both studies focusedon the Reading Recovery (RR) program out of New Zealand, whichincludes the use of DI for directed reading within academic settings(Clay, 1987). The RR program is designed to be an intensive, one-to-one tutoring program for those identified to be at risk for literacydifficulties. However, like other programs, RR is an early interventionprogram that aims to target those with literacy difficulties within thefirst-year of schooling (Reynolds & Wheldall, 2007). One such studycompared an RR intervention that was implemented for 30 min eachschool day for the duration of 1 year, against Project READ which isa method that emphasizes phonics (Acalin, 1995). Students in theProject READ condition received instruction in groups of 2 to 5 stu-dents for 30 min each school day for 1 year delivered by a resourcespecialist. Acalin (1995) found no significant difference between thosewho received one-to-one instruction when compared to those in thegroup-based intervention. These results parallel those of Evans (1996)who found no advantage for one-to-one interventions compared withgroup-based tutoring programs.

A more recent study by Vaughn et al. (2003) also emphasized theutility of small group instruction for reading comprehension and fluen-cy. Individual instruction was compared to group-based instruction,

1178 J. Harper, F. Schmidt / Children and Youth Services Review 34 (2012) 1176–1182

which consisted of groups of one teacher to three students, and oneteacher to ten students. This study found that small groups consistingof nomore than three second grade students were comparable to directone-to-one instruction for supplemental reading; however, those in thelarger groups displayed less significant improvements than individualor small group instruction (Vaughn et al., 2003). Though preliminary,research has generally found comparable results for individual andgroup basedmodels for remedial programs aimed at improving literacywhen group size is kept to a small number.

1.4. Tutoring by volunteers

Economically and logistically there has been an increasing em-phasis to use volunteers in tutoring programs. This has coincidedwith the America Reads Challenge initiative introduced by PresidentClinton in 1997 (Ritter, Barnett, Denny, & Albin, 2009). A recent sys-tematic review and meta-analysis of tutoring conducted by volun-teers was undertaken to provide greater clarity as to the effectivenessof volunteer tutoring programs on improving academic abilities amongchildren in elementary school (Ritter et al., 2009). Ritter et al. (2009)synthesized results of 21 randomized field trails published between1985 and 2005 that included adult volunteers who tutored primaryschool-aged children in grades K through 8. Adult, nonprofessionalvolunteer tutors were found to be effective at improving readingand language skills, specifically in the domains of reading and spell-ing (Ritter et al., 2009). Ritter et al. (2009) found small to moderateand statistically significant mean effect sizes on global reading(d=0.26), reading letters and words (d=0.41), reading oral fluency(d=0.30), and spelling (d=0.45). Furthermore, positive but non-significant effects were found for reading comprehension and math-ematics (Ritter et al., 2009). These encouraging results suggest thatvolunteer tutors can be effective in assisting children with remedialacademic programming.

1.5. Cultural considerations of remediation programs

It is estimated that Aboriginal children comprise close to 80%of the children living in out-of-home care, including foster care,group care, and institutional care in some Canadian provinces (Trocme,Knoke, & Blackstock, 2004). These statistics clearly attest to the over-representation of Aboriginal children in the child welfare system. Thereare limited studies that address discrepancies in academic achievementbetween Aboriginal and non-Aboriginal children in care. One study,however, found that non-Aboriginal children and youth scored higherin reading, writing and numerical skills than Aboriginal youths ingrades 4, 7 and 10 (Mitic & Rimer, 2002). In addition, Aboriginal chil-dren who were children in continuing care (CCC) had the lowestrates of reading, writing, and numeracy scores on the FoundationSkills Assessment (FSA) (Mitic & Rimer, 2002).

Remediation programs that have been developed and implemen-ted within Aboriginal or Indigenous cultures are sparse. An Australiastudied examined the effectiveness of DI among Aboriginal children.Maggs and Moore (1978) found that 74% of participants, Aboriginaland non-Aboriginal children, achieved true gain scores of 6 or moremental age months on the Peabody Picture Vocabulary Test. Accord-ing to the authors, Aboriginal children exceeded what is normallyachieved in terms of rates of academic skill development while re-ceiving DI (Maggs & Moore, 1978). The gains were significant forboth the Aboriginal children and their non-Aboriginal counterpartsfor measures of comprehension and vocabulary (Maggs & Moore,1978). Based on the preliminary results of DI among minority andAboriginal students, and due to the large representation of Aborigi-nal youth in the child welfare system, further attention and researchof this issue is warranted. The extension and effectiveness of agroup-based DI program aimed at improving academic achievementin Aboriginal children, or all foster children, is clearly lacking in the

literature. Therefore, DI was not only selected because of its demon-strated effectiveness (Adams & Engelmann, 1996; Borman et al., 2003),but also due to its cost-effectiveness of delivery in a group formatby volunteers and the preliminary effectiveness of DI for Aboriginalpopulations.

2. Current study

The present study sought to evaluate the effectiveness of a smallgroup-based DI tutoring program for children in foster care. Al-though past research has demonstrated the gravity of the problemand low academic achievement of children in foster care, few stud-ies have rigorously evaluated potential treatments. This two-yearstudy was designed to build on the small body of research on indi-vidually based DI for children in care (Flynn et al., -2012-this issue),by replacing individual tutoring with a small group instruction for-mat (i.e., four children per group). In addition, this study evaluatedthe effectiveness of using university student volunteers to deliverthe program. The TYCW program was chosen because the formatis not based on a school wide or systemic change in teaching envi-ronments and can be done in a small group-based tutoring formatoutside of regular school hours. It was expected that foster childrenwho received the TYCW intervention would exhibit a significantgain in reading, spelling, and mathematic skills, as measured bythe Wide Range Achievement Test Fourth Edition (WRAT-4). Thecurrent report details the first year results of the two yearevaluation.

3. Methodology

3.1. Participants

Children were eligible to participate if they were in foster or kin-ship care, behind in their academic achievement but not intellectuallychallenged (i.e. IQ>70), and were able to remain in the study for thefull 25 weeks of the intervention. Children were also required to bein grades 2 to 8, inclusive, and exhibited sufficient behavioural con-trol to participate in a small group instruction format, based on caseworkers clinical judgment. Given the unique cultural make-up ofthe local community, a majority of the participating foster childrenwere of Aboriginal background. During the first year of program im-plementation, 68 youth in grades 2 through 8 were referred by theirchild welfare case workers to participate. Table 1 summarizes the de-mographic characteristics of the 68 children at baseline for year one.Participants were between the ages of 6 and 13 (M=10.1 years,SD=1.7) and in grades 2 through 8 (M=5.1, SD=1.7). Thirty-threefoster children were randomly assigned to receive the group-basedtutoring intervention (57.6% male, 75.8% Aboriginal) and 35 to thecontrol group (57.1% male, 71.4% Aboriginal). Randomization, car-ried out by the program Research Randomizer, occurred with all re-ferrals prior to any direct contact or standardized testing with theyouth. Therefore, groups were non-equivalent at baseline as twochildren were inappropriately referred to the program, but this wasnot discovered until after random assignment. One youth displayedsuperior academic skills and did not require tutoring while a secondyouth had such severe behavioural difficulties that baseline testingwith the WRAT-4 could not be completed. Neither youth com-menced the study.

3.2. Measures

The WRAT-4 was used to measure academic achievement acrossfour dimensions including reading, spelling, sentence comprehensionandmathematics. Numerous validity studies have been carried out onthis academic measure. The WRAT-4 has been frequently used inmeasuring students' reading and math skills and provides population

Table 1Demographic characteristics of the 68 youth originally referred to the study.

Frequency Percent

Total sample(N=68)

Age (in years) 6 2 2.97 2 2.98 10 14.79 11 16.210 11 16.211 15 22.112 15 22.113 2 2.9

Ethnicity Caucasian 18 26.5Aboriginal 50 73.5

Grade 2 4 5.93 10 14.74 11 16.25 12 17.66 13 19.17 15 22.18 3 4.4

Gender Male 39 57.4Female 29 42.6

Primary placements sincelast admission in care (#)

0 13 19.11 27 39.72 16 23.53 2 2.94 3 4.45 3 4.46 1 1.5

Longest stable foster home(years)

Less than a year 5 7.91–2 years 6 9.52–3 years 2 3.23–4 years 11 17.54–5 years 7 6.35–6 years 9 14.36–7 years 6 9.57–8 years 7 11.18–9 years 5 7.99–10 years 5 7.910–11 years 3 4.8Missing 5

1179J. Harper, F. Schmidt / Children and Youth Services Review 34 (2012) 1176–1182

based comparison on academic achievement levels. The WRAT-4 isstandardized and norm-referenced, with all scores converted tostandard scores with a mean of 100, and a standard deviation of 15.A limitation, however, is that the WRAT-4 has not been validatedamong the Aboriginal Canadian population. As there are two alterna-tive forms (Blue and Green), the Blue Form was used at pre-test(September–October, 2010), while the Green Form was used atpost-test (May, 2011), with all scores expressed as standard scores.The WRAT-4 test manual reports internal consistency coefficientsranging from .87 to .96, and both high convergent and concurrentvalidity (Wilkinson & Robertson, 2006).

As previously stated, the WRAT-4 has four subtests includingWord Reading, Spelling, Sentence Comprehension, and Math Compu-tation. Word Reading measures letter and word decoding by wordrecognition and identification. Spelling measures the ability to encodesounds into written form by use of a dictated spelling format contain-ing both letters and words. Sentence Comprehension measures theability to gain meaning from words and to understand and compre-hend ideas and information within the sentences. Math Computationmeasures the ability to perform and execute mathematical computa-tions by counting, identifying numbers, solving simple oral problemsand calculating writtenmath problems. In administering theWRAT-4,the order was as follows: Spelling, Reading, Sentence Comprehension,and Math Computation. The foster children's WRAT-4 standard scoreswere computed from their raw scores and indicated how their levelof educational achievement compared with the standard scores(M=100, SD=15) of children of the same chronological age in thegeneral population.

3.3. Procedure

The implementation and evaluation of the group-based DI tutoringintervention project is a multi-year process. The first year involved theformation of a steering committee (2009–2010) which included front-line and senior management representation from both Aboriginal andnon-Aboriginal child welfare organizations in the community. Thecurrent authors were brought on board to assist in the evaluationof the intervention, which was planned to occur over a two-year pe-riod. Planning meetings occurred on a regular basis, every two tothree weeks, prior to the beginning of the first year of interventionin September, 2010. Dr. Robert J. Flynn and Michael Maloney werealso included on some of the first year planning meetings as consul-tants with information sessions being held with child welfare caseworkers, foster parents, and other community stake holders. In ad-dition, ethical approval was obtained from the local university Re-search Ethics Board prior to the commencement of the study.

Children in long-term out of home care referred by their childwelfare case-workers were randomly assigned to either the wait-listcontrol or tutoring DI conditions. All case-workers and foster care-givers of both intervention and control youth received an overviewof the program and were invited to attend one of several meetingsthat explained the program in greater detail. Children were requiredto assent to participate, while both child welfare case-workers andfoster caregivers provided consent for each youth's participation inthis study. Once participants were identified and the necessary in-formed consent obtained, baseline assessmentmeasureswere obtained,including demographic information.

3.3.1. Group based tutoringWithin the tutoring intervention program, children were assessed

on a measure of word fluency, used in the TYCW program, and placedinto small tutoring groups of three or four children according to skilllevel. The group-based tutoring groups ran over a 25-week timeframe, from September 2010 until April 2011, for 2 h each week,with either one or two tutor volunteers running each group. Eachsession followed the basic structure of Michael Maloney's TYCWcurriculum, which uses DI and behaviour management to improvethe educational attainment of children.

Volunteer university students were recruited to run the weeklytutoring program. Prior to working with the children, the tutors com-pleted two full days of training with the tutoring developer, MichaelMaloney. Tutors were required to collect performance data at eachtutoring session. This data comprised the fidelity checks and consistedof sound fluency (e.g. number of sounds read from a list of sounds per30 s), word fluency (e.g. number of words read from a list per 30 s),and story fluency (e.g. number of words read from a story in 1 min).This data was compiled into a weekly spreadsheet that was sent tothe Mr. Maloney. Throughout the course of the study, tutors had theirperformancemonitored byMichael Maloney who served as an ongoingconsultant. All volunteers received an honorarium at the middle andend of the tutoring program.

Participants in both the wait-list and intervention groups had theiracademic abilities assessed on two occasions, prior to and followingthe completion of the program, using the WRAT-4. The current resultsrepresent only a partial report of the data being collected for this twoyear evaluation. Threemonth follow-up academic scores and additionalmeasures of inattention, behavioural adjustment, and classroom aca-demic functioning will be presented in future reports following thecompletion of the second year of the study.

4. Analysis

All data collected was analyzed using SPSS version 19.0. Frequen-cies were analyzed to ensure that data was free of errors. Histogramswere generated to check the subscales of the WRAT-4 at pre- and

Table 2Adjusted post-intervention means of WRAT-4 subscales conditions holding Pre-intervention means constant.

Control (n=35) Intervention (n=30)

M M

Word reading 89.36 93.81Spelling 91.79 96.68Math 79.69 82.89Sentence comprehension 91.75 92.61

1180 J. Harper, F. Schmidt / Children and Youth Services Review 34 (2012) 1176–1182

post-intervention for outliers, skewness and kurtosis. A log transfor-mation was conducted on the baseline sentence comprehension sub-scale of the WRAT-4, to reduce skewness; however this did not yieldan improvement so the baseline sentence comprehension mean stan-dard scores were left untransformed. Therefore, all pre- and post-subscales, except for baseline scores of sentence comprehension,were within acceptable limits of normalcy.

5. Results

5.1. Equivalence of intervention and control groups at pre-test

There were no statistically significant differences (p>.05) betweenthe intervention and wait-list control groups at pre-intervention interms of gender, ethnicity, age (intervention group: M=9.79, SD=1.83; control group: M=10.37, SD=1.54), and the subtests of theWRAT-4. Therefore, the control and intervention groups were effec-tively randomized.

5.2. Attrition

From pre- to post-testing, three of the 33 (9.1%) foster children inthe tutoring intervention group and none of the control group youthwithdrew from the study. This attrition, however, did not reduceequivalence across group conditions. The intervention (n=30) andwait-list (n=35) groups were still equivalent in terms of gender, eth-nicity, age, and subtests of the WRAT-4 (p>.05).

5.3. Outcome results

In order to take into account potential clustering effects among thedifferent tutoring groups, multilevel modeling was used (Tabachnick& Fidell, 2007). Essentially, variance in student response to the tutor-ing intervention may be a function of the program (i.e., TYCW), butmay have also been affected by differences that naturally occur acrosseach small tutoring group. Multilevel modeling takes into accountthese dependencies by estimating the average response associatedwith each tutoring group (intercepts) as well as group differences inassociations (slopes) between predictors andoutcomemeasures. Condi-tion (intervention andwait-list control) and baseline academic achieve-ment scores (reading, sentence comprehension, spelling and math)were entered as fixed effects in the multilevel model. For all post-intervention academic outcome variables, entering the predictoras a group rather than as an individual observation did not improvethe fit of the model1. The use of multi-level modeling did not im-prove or add to statistical analyses. Therefore, all analyses in the prelim-inary year one data were evaluated using an ANCOVA model, ignoringclustering of tutoring groups.

An analysis of covariance (ANCOVA) with condition (control vs.intervention) as the between subject variable and WRAT-4 post-intervention outcome scores as the within-subject variable was com-pleted. The pre-test WRAT-4 academic scores were entered as a covar-iate, via multiple regression, in order to control for the impact thatpre-existing baseline differences on academic abilities could haveon post-intervention academic outcomes. Due to attrition of threechildren in the intervention condition, analyzes were performed basedon the 65 youth who completed the study from baseline to post-intervention.

There was a significant effect of condition on reading, F(1, 62)=10.58, p=.002. The adjusted marginal means, as displayed in Table 2

1 For instance using reading as the outcome, the group model did not differ signifi-cantly from the model where all participants were considered as individual observa-tions, χ2 (1, N=65)=402.99 - 400.3326=2.664, p>.05. There was no significantimprovement in the fit of the model when the intercept, slope, and intercept and slopewere allowed to be random and by tutoring group.

and the unadjusted mean standard scores in Fig. 1, show that thosewho received the tutoring intervention had significantly higher wordreading scores on the WRAT-4 relative to their peers in the wait-listcondition, even when adjusted for pre-intervention scores. In accor-dance with the recommendations of the What Works Clearinghouse(WWC, 2008) Procedures and Standards Handbook (version 2.0),Hedge's g was used to compute effect sizes that were unbiased by asmall sample size. The effect size for reading, g=0.42, was above thethreshold of 0.25 that WWC (2008) considers “substantively impor-tant”. This suggests that the DI tutoring program had a small to moder-ate effect on word reading. Furthermore, as a means of illustrating theimpact of the tutoring, an improvement index was calculated from theHedges' g (WWC, 2008, p. 24). The improvement index is conceptual-ized as the difference between percentile rank related to the tutoringgroup mean and the percentile rank corresponding to the controlgroup mean, which is defined at the 50.0th percentile. The improve-ment index for reading, calculated from Appendix Z in Howell (2002),was 16.3% (e.g. 66.3th–50.0th percentiles). In other words, the averagefoster child in the tutoring condition was at the 66.3th percentile of thecontrol group, whose average foster child as at the 50.0th percentile.

Preliminary year one analysis also revealed a significant effect ofcondition on spelling, F(1, 62)=8.78, p=.004. The adjusted marginalmeans, as displayed in Table 2, and the unadjusted mean standardscores in Fig. 2, show that those who received the tutoring interven-tion had significantly higher spelling scores on the WRAT-4 relativeto those in the wait-list control condition, when adjusted for pre-intervention spelling scores. The Hedge's g of 0.38 for spelling im-provement falls above the WWC threshold of 0.25 and is thereforeconsidered substantively important and a small to moderate effectsize. The improvement index scorewas 14.8% (64.8th–50.0th percentile).

There was no statistically significant effect of condition on sentencecomprehension, F(1,60)=.45, ns, or mathematics, F(1,62)=2.12, ns,after adjustment for covariate of baseline mean standard scores. Formathematics, the Hedge's g of .26 was still substantively importantand fell within the small to moderate range. The improvement indexwas 10.3% (60.3th–50.0th percentile). However, improvements in sen-tence comprehension, while in favour of the intervention group, was

Fig. 1. Mean standard score differences on word reading.

Fig. 2. Mean standard score differences on spelling.

1181J. Harper, F. Schmidt / Children and Youth Services Review 34 (2012) 1176–1182

not statistically meaningful or important, g=.095. The improvementindex was 3.6% (53.6th–50.0th percentile).

6. Discussion

This study investigated the effectiveness of a 25-week group-based DI program on academic achievement among youth in fostercare. Overall, the foster children who participated in this study pos-sessed average level spelling skills, but below average reading andsentence comprehension skills, and very low math skills as evidencedby standardized scores. Despite the small sample size, the group-based DI intervention had statistically significant and positive effectson reading (g=.42) and spelling (g=.38) skills that exceeded thoseattained from the normal course of development and schooling by thefoster children in the matched control group. While not statisticallysignificant, the effect size for math skills, g=.26, fell within the sub-stantively important range. This suggests that the foster children ex-perienced a positive impact and improvement in their reading andspelling through the tutoring intervention. However, the effect sizerelated to sentence comprehension was not meaningful and suggeststhat the tutoring intervention, as delivered in this format, may not behelpful for foster children in this area. This preliminary finding willneed further study in order to better understand the potential limita-tions of DI for the abilities assessed by sentence comprehension onthe WRAT-4.

The current results are consistent with the positive conclusionsreached by Ritter et al. (2009) in their systematic review and meta-analysis of tutoring by adults, nonprofessional volunteers, which in-cluded college-age students, comparable to the volunteer undergrad-uate university students in the current study. The magnitude of thestatistically significant effects found in the current study matchedthose reported by Ritter et al. (2009), but exceeded those reportedby Shippen et al. (2005). Across various measures of reading, Ritteret al. (2009) found small to moderate and statistically significant ef-fect sizes, using Hedge's g, ranging from .18 to .42. These results areconsistent with the effect size of 0.42 found for reading in the currentstudy. It may be that the small effect size (d=.24) reported byShippen et al. (2005) may reflect the reduced duration of their DI pro-gram, lasting only six weeks in duration when compared to 25 weeksin this intervention. The pooled effect size of .45 for spelling found inthe Ritter et al. (2009) study was slightly higher than the 0.38 Hedge'sg obtained in the current study, however, both are within the moder-ate range.

As the current study was modeled after the work of Flynn et al.and used the same DI instruction program, a comparison of outcomeresults is meaningful. Flynn et al. (2012-this issue) found significantimprovements in mathematics and reading comprehension, which

is synonymous with sentence comprehension in this study, when fos-ter children were instructed in a one-to-one format by foster parents.These were the two academic domains not statistically significant inthe current study. This discrepancy may be accounted for by the useof a different instruction model (ie., individual vs. group format) ormay be related to some additional factors unknown at this time. Forinstance, Flynn etal. (2012-this issue) included 30 min of interactivereading between child and caregiver in addition to the direct instruc-tion component. As sentence comprehension is a higher order pro-cess requiring both reading and comprehension, includinginteractive reading time with an older adult or caregiver may en-hance the benefits of direct instruction and increase the benefits onthe sentence comprehension subscale. Given the limited research ofDI with foster children, further investigation will be needed. Howev-er, it is important to note that Ritter et al. (2009) reported similar re-sults to the current study in the domains of sentence (reading)comprehension and mathematics, with non-significant effect sizes of.18 and .27, respectively. One important explanation for the weakereffects for math skills in this study, when compared to Flynn et al.(2012-this issue), relates to the weak implementation of the mathcomponent of the TYCW program in this study. There were difficul-ties with the volunteer tutors delivering the math component of theDI program that were not identified and corrected until the firstyear of tutoring was well underway. A possible explanation is thatthe children in the tutoring condition did not receive an adequatedosage of mathematics tutoring, which may have caused less of asignificant improvement in this academic domain.

6.1. Lessons learnt

The current results are preliminary and represent the first year ofprogram implementation. A number of lessons have been learnedover the course of this first year and are actively being incorporatedinto the second year of the intervention. For example, a critical lessonwas the need to provide more extensive training to the tutor volun-teers with a particular emphasis on completing fidelity checks andimplementation of the behavioural management reward system. Inthe current study, the emphasis of initial training was on readingand literacy, with less devoted to the computer based math compo-nent and little on behaviour management and the implementationof the reward system. Furthermore, as there was limited informationprovided during training as to how to report fluency checks, thismeant that it was difficult to determine which tutors were imple-menting the program with high fidelity versus those who were not.This may have reduced the dosage and/or balance of skills taughtacross each tutoring group and inadvertently reduced the actual ef-fect size of the program across academic skill areas. Therefore, withfuture training sessions, more time should be devoted towards thepragmatics and recording of fluency checks.

Furthermore, as there was no condition controlling for the effectsof being placed in a structured program, it is difficult to distinguishbetween possible training effects (e.g. DI) from the effects of receiv-ing structure and support for 25 weeks. As the current study repre-sents preliminary year one outcome results, additional investigationon the potential benefits of DI group-based programs are warranted.

6.2. Future directions

Following the completion of year two of this study, the sample sizewill be increased and allow for additional analyses. Not only will thisprovide greater statistical power to assess the effect sizes found dur-ing the first year, but also allow for a more detailed examination ofpotential moderators. For example, inattention, placement stabilityand ethnicity may play a role in moderating the influence of thetutoring program on academic achievement scores. These questionswill be studied and more fully understood after completion of the

1182 J. Harper, F. Schmidt / Children and Youth Services Review 34 (2012) 1176–1182

second year of the study. In particular, since the participant sampleconsisted largely of Aboriginal children, the relative impact of cultureon academic outcomes can be more clearly determined. However,based on the current results, which consisted of a sample of bothAboriginal and non-Aboriginal children, DI appears to be a promis-ing instruction method. It will also be important to determine if thedemonstrated gains on standardized achievement tests, such as theWRAT-4, are generalized to the classroom setting and school aca-demic performance. Information on this latter issue is critical inorder to demonstrate generalizability of the intervention effects toeveryday life.

The results generated from this academic intervention study haveimplications for both research and government stakeholders whostrive to generate efficient and effective policies and curriculums,especially for those students with academic deficits. The prelimi-nary effectiveness of this group-based DI model provides an addi-tional cost-effective avenue for agencies to explore as a means ofimproving reading and spelling for foster children who are behindacademically. Although the results from this study need further in-vestigation and replication, it is clear that those youth who receivedtutoring in the current study demonstrated significant improvementsin reading and spelling scores relative to their same aged peers in thecontrol condition. This is an encouraging finding which can make alife-long improvement in a vulnerable population of children if theproper resources are allocated to this need.

Acknowledgements

We would like to thank the Thunder Bay Crown Ward Champion-ship team for their support of this program and the funding providedthrough the Ministry of Children and Youth Services, Ministry ofEducation, and the Ministry of Training, Colleges and Universities.We also acknowledge the full participation of the Children's Aid So-ciety for the District of Thunder Bay and Dilico Anishinabek FamilyCare. Thanks to Bruce Weaver and Dwight Mazmanian who helpedwith quantitative data analysis, and to Robert J. Flynn and MichaelMaloney who gave assistance regarding the structure and imple-mentation of the tutoring program.

References

Acalin, T. A. (1995). A comparison of reading recovery and project READ. MastersAbstract International, 33(06), 1660 (UMI No.1361908).

Adams, G., & Engelmann, S. (1996). Research on direct instruction: 25 years beyondDISTAR. Seattle, WA: Educational Achievement Systems.

Ashworth, D. R. (1999). Effects of direct instruction and basal reading instruction pro-grams on the reading achievement of second graders. Reading Improvement, 35,150–156.

Borman, G. D., Hewes, G. M., Overman, L. T., & Brown, S. (2003). Comprehensive schoolreform and achievement: A meta-analysis. Review of Educational Research, 73,125–230, doi:10.3102/00346543073002125.

Burley, M., & Halpern, M. (2001). Educational attainment of foster youth: Achievementand graduation outcomes for children in state care. Olympia, WA: Washington StateInstitute for Public Policy.

Casey, K., Hagaman, J., Trout, A. L., Reid, R., Chmelka, B., & Thompson, R. (2008). Childrenwith ADHD in residential care. Journal of Child and Family Studies, 17, 909–927, doi:10.1007/s10826-008-9198-x.

Chamberlain, P., Moreland, S., & Reid, K. (1992). Enhanced services and stipends forfoster parents: Effects of retention rates and outcomes for children. Child Welfare,71(5), 387–401.

Clay, M. M. (1987). Implementing Reading Recovery: Systematic adaptations to an educa-tional innovation. New Zealand Journal of Educational Studies, 22(1), 35–58.

Cole, K. N., Dale, P. S., Mills, P. E., & Jenkins, J. R. (1993). Interaction between early in-tervention curricula and student characteristics. Exceptional Children, 60, 17–28.

Courtney, M., Piliavin, I., Grogan-Kaylor, A., & Nesmith, A. (2001). Foster youth transi-tions to adulthood: A longitudinal view of youth leaving care. Child Welfare, 80,685–717.

Dolezal, D. N., Weber, K. P., Evavold, J. J., Wylie, J., & McLaughlin, T. F. (2007). The effectsof a reinforcement package for on-task and reading behavior with at-risk and mid-dle school students with disabilities. Child & Family Behavior Therapy, 29, 9–25, doi:10.1300/J019v29n02_02.

Dowdell, T. (1996). The effectiveness of direct instruction on the reading achievement ofsixth graders. ERIC DOCUMENT REPRODUCTION service, No. ED 396268.

Elbaum, B., Vaughn, S., Hughes, M. T., & Moody, S. W. (2000). How effective are one-to-one tutoring programs in reading for elementary students at risk for readingfailure? A meta-analysis of the intervention research. Journal of Educational Psy-chology, 92, 605–629, doi:10.1037/0022-0663.92.4.605.

Evans, T. L. P. (1996). “I can read Deze books!”: A qualitative comparison of the readingrecovery program and a small-group reading intervention. Dissertation AbstractsInternational, 57(02), 565A (UMI No. 9619200). Retrieved June 15, 2010, from Dis-sertations and Theses database.

Felton, R. H. (1993). Effects of instruction on the decoding skills of children withphonological-processing problems. Journal of Learning Disabilities, 26, 583–589,doi:10.1177/002221949302600904.

Flynn, R. J., & Biro, C. (1998). Comparing developmental outcomes for children in carewith those for other children in Canada. Children & Society, 12, 228–233, doi:10.1111/j.1099-0860.1998.tb00070.x.

Flynn, R. J., Ghazal, H., Legault, L., Vandermeulen, G., & Petrick, S. (2004). Use of popu-lation measures and norms to identify resilient outcomes in young people in care:An exploratory study. Child and Family Social Work, 9, 65–79, doi:10.1111/j.1365-2206.2004.00322.

Flynn, R. J., Marquis, R. A., Paquet,M. -P., Peeke, L. M., & Aubry, T. D. (2012). Effects of individ-ual direct-instruction tutoring on foster children's academic skills: A randomized effec-tiveness trial. Children and Youth Services Review, 34, 1183–1189 (this issue).

Howell, D. C. (2002). Statistical methods for psychology (Fifth Edition). Belmont, CA:Duxbury.

Maggs, A., & Moore, J. (1978). Teaching academic skills by direct instructional methodsto school aged Aboriginal and non-Aboriginal children with learning problems. TheExceptional Child, 25, 127–135, doi:10.1080/015655780250205.

Mitic, W., & Rimer, M. -L. (2002). The educational attainment of children in carein British Columbia. Child & Youth Care Forum, 31, 397–414, doi:10.1023/A:1021158300281.

Reynolds, M., & Wheldall, K. (2007). Reading recovery 20 years down the track: Look-ing forward, looking back. International Journal of Disability, Development and Edu-cation, 54, 199–223, doi:10.1080/10349120701330503.

Ritter, G. W., Barnett, J. H., Denny, G. S., & Albin, G. R. (2009). The effectiveness ofvolunteer tutoring programs for elementary and middle school students: A meta-analysis. Review of Educational Research, 79, 3–38, doi:10.3102/0034654308325690.

Ryder, R. J., Burton, J. L., & Silberg, A. (2006). Longitudinal study of direct instruction ef-fects from first through third grades. The Journal of Educational Research, 99,180–191, doi:10.3200/JOER.99.3.179–192.

Shippen, M. E., Houchins, D. E., Steventon, C., & Sartor, D. (2005). A comparison of twodirect instruction reading programs for urban middle school students. Remedialand Special Education, 26, 175–182, doi:10.1177/07419325050260030501.

Stahl, S., Duffy-Hester, A., & Stahl, K. (1998). Everything you wanted to know aboutphonics (but were afraid to ask). Reading Research Quarterly, 33, 338–355, doi:10.1598/RRQ.33.3.5.

Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (fifth edition). Boston,MA: Pearson Education.

Torgesen, J., Morgan, S., & Davis, C. (1992). Effects of two types of phonological aware-ness training on word learning in Kindergarten children. Journal of Educational Psy-chology, 84, 364–370.

Torgesen, J. K., Wagner, R. K., & Rashotte, C. A. (1997). Prevention and remediation ofsevere reading disabilities: Keeping the end in mind. Scientific Studies of Reading,1, 217–234, doi:10.1207/s1532799xssr0103_3.

Trocme, N., Knoke, D., & Blackstock, C. (2004). Pathways to the overrepresentation ofAboriginal children in Canada's child welfare system. The Social Service Review,78, 577–600, doi:10.1086/424545.

Trout, A. L., Hagaman, J., Casey, K., Reid, R., & Epstein, M. H. (2008). The academic statusof children and youth in out-of-home care: A review of the literature. Children andYouth Services Review, 30, 979–994, doi:10.1016/j.childyouth.2007.11.019.

Vaughn, S., Linan-Thompson, S., Kouzekanani, K., Bryant, D. P., Dickson, S., & Blozis, S. A.(2003). Reading instruction grouping for students with reading difficulties. Reme-dial and Special Education, 24, 301–315, doi:10.1177/07419325030240050501.

What Works Clearinghouse (2008). Procedures and standards handbook (version 2.0).Washington, DC: Institute of Educational Sciences.

Wilkinson, G. S., & Robertson, G. J. (2006). Wide Range Achievement Test 4 (WRAT4).Wilmington, DE: Jastak Association.