teaching the language of wider communication, minority ......teaching the language of wider...

Teaching the Language of Wider Communication,Minority Students, and Overall EducationalPerformance: Evidence from a RandomizedExperiment in Qinghai Province, China

fang laiKatholieke Universiteit Leuven and Center for Chinese Agricultural Policy,Institute of Geographic Sciences and Natural Resource Research,Chinese Academy of Sciences

linxiu zhang, qinghe qu, and xiao huCenter for Chinese Agricultural Policy, Institute of Geographic Sciences andNatural Resource Research, Chinese Academy of Sciences

yaojiang shiShaanxi Normal University

matthew boswell and scott rozelleStanford University

The education of poor and disadvantaged populations, particularly those fromminority subgroups, has been a long-standing challenge to education systemsin both developed and developing countries ðe.g., World Bank 2001, 2004;Glewwe and Kremer 2006; Planty et al. 2008Þ. For example, over the past de-cade in the United States the high school dropout rate of Hispanic students hasremained at least twice as high as that of white students ðAud et al. 2011Þ. Usingdata from the German Microzensus and the German Socio-Economic Panel,Alba, Handl, and Müller ð1994Þ found that, relative to young Germans withidentical sociodemographic characteristics, Italian, Turkish, and Yugoslav chil-dren are overrepresented in the lowest academic track of the German schoolsystem.One reason for the underperformance of ethnic minority students might be

low competency in the national language or the language of wider communi-

We would like to acknowledge Acer Computers and the Asia-Pacific Economic Cooperation ðAPECÞDigital Opportunity Center ðADOCÞ 2.0, and the Natural Science Foundation of China (grant71333012), for their generous support for the Rural Education Action Program’s Technology andHuman Capital theme area. The hard work of dozens of volunteers from the Chinese Academy ofSciences, Northwest University of Xi’an, and Qinghai Minorities University made this article possible.

Electronically published May 4, 2015© 2015 by The University of Chicago. All rights reserved. 0013-0079/2015/6304-0004$10.00

cation ðLWCÞ. Studies worldwide have documented that language difficultiesnegatively affect the academic performance of students whose mother tongueis not the LWC. For example, August and Shanahan ð2006Þ and Genesee et al.ð2006Þ show that language barriers were among the major factors for the rel-ative low performance of non-English-speaking minority groups in the UnitedStates. As a result, improving national language skills has become one of themost urgent academic needs for ethnic minority groups or immigrants/refu-gees whose mother tongue is not the LWC.Usually, there are two ways to deal with the language barriers faced by

minority students in their academic progress. One way is to implement bilin-gual education and teach in the mother tongue of the minority group in theearly years of their education to secure adequate academic progress. In the bi-lingual education regime the instruction language is gradually switched to theLWC in later years of schooling. The other approach is keeping the LWC asthe language of instruction while making efforts ðe.g., through specially targetedcoursesÞ to improve the LWC skills of the students who are nonnative LWCspeakers ðhenceforth, keeping instruction in the LWCÞ.While there have been studies on the effectiveness of these two approaches

ðbilingual education and keeping instruction in the LWCÞ, the findings areinconclusive. Although the overall findings from these studies and an 8-year-oldmeta-analysis of these studies are in favor of bilingual education ðBamgbose1984; Yates 1995; IDRC 1997; Cheung and Slavin 2005Þ, most recent large-scale and long-term evaluations do not find significant benefits of bilingualeducation over keeping instruction in the LWC programs ðSlavin et al. 2009,2011Þ. Recent studies agree that high-quality program design is much moreimportant than the language of instruction ðAugust and Shanahan 2006; Slavinet al. 2011Þ. In other words, it does not really matter whether a school systemoffers bilingual education or keeps instruction in the LWC; it is the quality of thelanguage program that matters.While the existing literature is informative, there are several limitations.

First, most of the recentmethodologically rigorous studies directly compare therelative effectiveness of bilingual education versus LWC programs. There arefew recent studies that compare individual programs ðeither bilingual or LWCprogramsÞ with a pure control group, particularly within large-scale, random-ized controlled trials. Second, there is generally a dearth of methodologicallyadequate studies outside of theUnited States.Most studies in the literature havestudied bilingual or English-only ðor LWC-onlyÞ programs for English lan-guage learners, predominantly for Spanish speakers in the United States.Apart from these critical deficiencies in the existing research, there has also

been almost no rigorous investigation of the potential spillover/crossover ef-

754 E C O N O M I C D E V E L O P M E N T A N D C U L T U R A L C H A N G E

fects of language-focused interventions on minority student performance insubjects other than language. Among the limited number of studies that areðeven remotelyÞ related to this topic, none have investigated the spillover ef-fects of language interventions on minority students directly. Master et al.ð2013Þ used multiyear data from New York City and an instrumental vari-able ðIVÞ approach ðthe IV was the 2-year lagged values of the endogenousvariable—teacher quality measured by “teacher value-added”Þ to demon-strate that “high-quality English teachers” have long-term positive impactsnot only on language performance but also on math performance. Employinga randomized controlled trial approach in India, He, Linden, and MacLeodð2008Þ also found positive spillovers onto student math performance from acomputer-based program that teaches students English. However, accordingto the authors’ analyses, the mechanism of the improved math performancewas not better English language skills of students but rather the result of moreeffective program time management by the teacher supervisors. To our knowl-edge, there has not been an in-the-field experiment designed explicitly toidentify the impact of a program that was created to improve LWC/nationallanguage skills and also examine the subsequent effects on overall academicperformance.The overall objective of this study is to examine the impact of an LWC

learning program on the overall academic performance ðand other outcomesÞ ofstudents who have limited competency in the LWC. To achieve this researchobjective, we present the results from a randomized field experiment of acomputer assisted learning ðCALÞ program that teaches rural ethnic minoritystudents LWC language skills and examines their performance in both LWCskills andmath. Specifically, theCALprogram teachesMandarin language skillsto non-Mandarin-speaking students in 57 schools in Qinghai Province innorthwestern China. In this part of Qinghai, around 86% of the students areethnic minorities whose mother tongues are not Mandarin. The academicperformance of non-Han ðminorityÞ students in this area of China lags sub-stantially behind Han ðnonminorityÞ students ðYang, Wang et al. 2013Þ.We are particularly interested in a CAL program because even though ex-

isting empirical studies have shown mixed evidence of the impacts of CAL onstudent academic outcomes, many of the recent randomized experiments haveshown that CAL can be particularly effective in meeting the academic needs ofstudents who live in environments in which schools and teaching are poor inquality and the home learning environment is inadequate ðe.g., Banerjee et al.2007; Barrow, Markman, and Rouse 2008; He et al. 2008; Lai et al. 2013,2014; Yang, Zhang et al. 2013Þ. For these students CAL may be able to act asa substitute for tutors or higher-quality in-school teaching.

Lai et al. 755

Although there have been other CAL studies in China and elsewhere in recentyears, the CAL program that forms the basis for this article is different fromthese. To our knowledge, the study in this article is the first CAL study con-ducted in China to focus on implementing a computer assisted Mandarinlanguage learning program for minorities and examining, in particular, itsimpact on both student Mandarin language skills and performance in othersubjects. In the studies in Lai et al. ð2013, 2014Þ andMo et al. ð2014Þ, theCALprograms were designed to deliver remedial tutoring for math, not the Man-darin language. These other CAL studies were also undertaken in regions ofChina that catered to Han students whose mother tongue was Mandarin—theofficially designated national language/LWC in China—and their samples in-cluded virtually no non-Han students ðLai et al. 2013, 2014; Mo et al. 2014Þ.In one paper ðYang, Zhang et al. 2013Þ—whichmight be considered a small

meta-analysis of the CAL programs that have been run in China over the pastseveral years—a set of coefficients from Qinghai that showed the impact of aMandarin CAL program on Mandarin language performance was presentedsimultaneously with two sets of coefficients from the Beijing and Shaanxi CALprogram sites. The focus of this article, however, was on the gender differ-ence of the impacts of CAL programs. There was no discussion of the spill-over effects from the Mandarin CAL program onto student overall academicperformance—the focus of the current study. The implications of these spill-overs are important to understand given chronic academic underperformanceamong nonnative speakers of Mandarin in comparison to native-speakingpeers. The only randomized controlled trial that has explored the spillovereffects of a CAL language program on student performance in other subjects isHe et al. ð2008Þ in India. However, as mentioned above, they explained im-proved math skills not through improved language skills but rather throughthe more efficient delivery of the language curriculum. Hence, the results fromthe current study are the first to demonstrate the effect of a CAL languageprogram on minority students’ academic performance in a school system thatis committed to keeping instruction in the LWC.In launching a study on the effect of a CAL program, we contribute to the

existing literature in several ways. First, by implementing a randomized con-trolled trial, our study not only provides an unbiased estimate of the impactsof a CAL language program on student language competency but also providesevidence of the causal impacts of improved language competency on studentacademic performance in other subjects ðin this case, mathÞ. Furthermore, thisstudy focuses on a CAL program teaching Mandarin to rural ethnic minoritystudents in China and, hence, is outside the usual environment ðteachingSpanish-speaking Latino children English in the United StatesÞ. Overall, our


work demonstrates that a Mandarin CAL language program has the potentialto provide a relatively quick and effective remedy for ðat least part ofÞ the severeunderperformance of rural minority children in China in both Mandarin andother subjects.The rest of the article is organized as follows. In the next section, we intro-

duce the sample, data, and method of this article. The following two sec-tions report the results and conclude.

Sampling, Data, and MethodSampling and the Process of RandomizationWe conducted a clustered randomized controlled trial of CAL in rural schoolsduring the spring semester of 2011. A total of 1,889 students in 57 elementaryschools in poor minority areas in China’s Qinghai Province participated in ourstudy. We adopted a three-step process to arrive at this sample.First, we chose our sample counties in a way that allows us to focus our

study on the impact of the CAL program on improving the Mandarin lan-guage skills of students in rural minority areas. In order to do so, we first choseQinghai Province, a poor, rural province. More than 60% of the populationlives in rural areas. Qinghai also has a large minority population. Ethnic mi-norities make up around 46% of the province’s population. The share of therural population that is minority is even higher. We then chose three countiesin Haidong Prefecture, Huzhu, Hualong, and Xunhua. Each of the threecounties in our sample has the characteristics that fit our research purpose:each county is poor, rural, and minority autonomous. Second, after choosingthe counties, we sampled 57 schools. To do so, we obtained a comprehensivelist of all schools. We called each school to confirm its number of minoritystudents and excluded schools with less than 20% minority students ðsinceminorities were the main focus of our studyÞ. For implementation conve-nience we also excluded schools that were either too big ðmore than 150 third-grade studentsÞ or too small ðless than 20 third-grade studentsÞ. There were70 schools that met the two criteria above. Because of our limited computerprovision, we then randomly chose 57 schools for our final sample.1

1 Our sample selection proceeded as follows. According to our original design, we needed30 treatment and 30 control schools. This was done on the basis of power calculations. We thenproceeded to randomly choose 60 schools from a list that were available in the three samplecounties. The list contained the school name, location, and the number of students in grade 3.From this information, we randomized the group into 30 treatment and 30 control schools.Before starting, however, it was discovered that three of the schools had been closed. Of the threeschools, twowere supposed to be assigned to the treatment school group; one was supposed to beassigned to the control school group. This means that the number of schools available for theexperiment was 57 ð28 in the treatment group and 29 in the control groupÞ. It was at this time

Lai et al. 757

Finally, because the Mandarin CAL software we used was only available atthe third-grade level, we only examined third-grade students in our sampleschools. All of the third-grade students in the 57 sample schools were includedin the study. In total, there were 1,889 third-grade students included in thesample ðfig. 1Þ, among whom 86% belonged to a minority ethnic group.After choosing the 57 schools for our sample, we randomly chose ðwithout

stratificationÞ 26 schools from these 57 schools to receive the CAL interven-tion. The 764 third-grade students in the 26 treatment schools constitutedthe treatment group ðfig. 1Þ. The 1,125 third-grade students in the other31 schools served as the control group.Using data from the treatment and control schools, we can show that the

sample is well balanced. To show this, we used a set of student and schoolcharacteristics to check the validity of the random assignment for our sam-ple by regressing each variable on the treatment dummy variable ðtable 1Þ.Specifically, the individual characteristics included measures of whether thestudents were male, their ethnic group ðHan, Hui, Salar, Tu, and TibetanÞ,county, whether they were the only child in the family, whether their father wasilliterate ðfather illiterateÞ, whether their mother was illiterate ðmother illit-erateÞ, and whether their parents were still farmers and worked on a farm ornot ðfamily off-farmÞ. The school-level characteristics included the numberof students, the teacher-student ratio, the proportion of female teachers, anindex of the availability of school facilities, and the proportion of studentsreceiving poverty subsidies.We found that the differences between the treatmentand control groups were statistically insignificant ðtable 1, cols. 5 and 6Þ. TheF-tests of joint significance of the individual characteristics and school char-acteristics also showed that there were no significant differences in the indi-vidual and school characteristics between the treatment and control groups.Although at the time of the baseline survey the main sample included a

total of 57 schools and 1,889 third-grade students, there was some attrition bythe end of the study. For various reasons ðmainly because of school transfersand absences due to illnessÞ, at the time of the evaluation survey we followed

that we decided how many computers were needed for the treatment schools. This number wascalculated as roughly half of the number of grade 3 students.When the computers were deliveredto the project site, it was discovered that there were an insufficient number of computers. Wethus randomly excluded two of the treatment schools from the treatment group in order to equipthe remaining treatment schools with enough computers. We used these two schools as controlschools in the analysis. This action served to reduce the number of treatment schools to 26 andincreased the number of control schools to 31. We calculated the power with our sample of57 schools. The power to detect a treatment effect of a size around 0.2 and significant at the 5%level is around 0.8, with an intraclass correlation coefficient ðICCÞ equal to 0.09 ðwhich is theactual ICC of our studyÞ, R2 equal to 0.5, and total number of clusters equal to 57 ðwith 30 ob-servations in each cluster on averageÞ.


up with 1,717 third-grade students in the 57 sample schools ðfig. 1Þ. Amongthe 764 treatment students, 680 remained in our sample at final evaluation.Among the 1,125 control students, 1,037 remained in our sample at final eval-uation.To understand the nature of those who attrited and assess whether the at-

trition affected the nature of the sample ðor the validity of the randomizationÞ,we regressed each variable from a set of student characteristics on a constant, adummy for attrition, a dummy for treatment, and an interaction of the attri-tion and treatment dummies. According to our findings ðtable 2Þ, we see thatthose who attrited ðcol. 2Þ have lower Mandarin scores, have a higher share ofmothers who were illiterate, were less likely to live with both parents, were lesslikely to be Tibetan and more likely to be Hui, and were more likely to be fromXunhua. However, as seen from the fact that none of the coefficients of theinteraction variables were statistically significant, there is no evidence that theattrition affected the validity of our randomization. Moreover, there is no sta-tistically significant difference in the attrition rate between the treatment andcontrol groups ðtable 1Þ.

Experiment Arms/InterventionsOur experiment focused fully on one treatment group ðthe third-grade stu-dents of the 26 treatment schoolsÞ and one control group ðthe third-grade

Figure 1. Experiment profile

Lai et al. 759

students of the 31 control schoolsÞ. The CAL Intervention was the onlyintervention going on in these schools at the time.

CAL Intervention Group ðGrade 3 Students in the 26 Treatment SchoolsÞThe intervention involved computer-assisted Mandarin remedial tutoringsessions, which were designed to complement the regular in-class Mandarin

TABLE 1COMPARISON OF STUDENT AND SCHOOL CHARACTERISTICS BETWEEN THE TREATMENT AND

CONTROL SCHOOLS (INCLUDING BOTH THE SAMPLE AND ATTRITIONS)

Treatmentð26 Schools,N 5 764Þ

Controlð31 Schools,N 5 1,125Þ

DifferenceðTreatment −

ControlÞMean SD Mean SD Mean SDð1Þ ð2Þ ð3Þ ð4Þ ð5Þ ð6Þ

Student-level characteristics:Baseline Mandarin score ðunits of SDÞa .01 1.03 .01 .97 .02 .16Baseline math score ðunits of SDÞa .05 1.02 .03 .98 .08 .14Male ð1 5 yes; 0 5 noÞ .47 .50 .50 .50 .02 .03Only child ð1 5 yes; 0 5 noÞ .13 .33 .14 .34 .01 .02Father illiterate ð1 5 yes; 0 5 noÞ .19 .39 .21 .40 .01 .03Mother illiterate ð1 5 yes; 0 5 noÞ .58 .49 .55 .50 .03 .04Family off-farm .07 .26 .07 .26 .00 .02Live with both parents .54 .50 .57 .50 .03 .05Ever used a computer ð1 5 yes; 0 5 noÞ .32 .47 .37 .48 .05 .07Access to other modern technologiesb .48 .22 .46 .24 .01 .03Tibetan .13 .34 .11 .31 .03 .06Tu .22 .41 .21 .41 .01 .09Hui .31 .46 .34 .47 .03 .12Salar .23 .42 .17 .38 .06 .11Han .10 .30 .17 .37 .07 .05Attrition .11 .31 .08 .27 .03 .02

School-level characteristics:Number of students 198.41 87.84 251.16 111.36 −52.75 31.76Teacher-student ratio .06 .02 .05 .01 .00 .00Proportion of female teachers .51 .21 .55 .18 .03 .06School facility indexc .57 .14 .57 .19 .00 .05Proportion of students receivingpoverty subsidy 82.87 71.76 86.31 67.48 −3.44 20.90

County Huzhu .35 .48 .42 .49 .07 .13County Hualong .25 .44 .18 .38 .08 .12County Xunhua .40 .49 .41 .49 .01 .14

Source. Authors’ survey.a Score on the standardized math/Mandarin test that was given to all students in the sample before thecomputer assisted learning ðCALÞ program.b Mean value of a set of 0/1 dummy variables including whether the student has used cell phone, Internet,game console, CAL software, and videos for learning assistance.c Mean value of a set of 0/1 dummy variables indicating the availability ð05 unavailable; 15 availableÞ ofsome school facilities, including enclosure, playground, reading room, library, running water, hot water,and clinic.


language curriculum for the spring 2011 semester. Under the supervision oftwo teacher-supervisors trained by our research group, the students in thetreatment group had two 40-minute CAL sessions per week during lunchbreak or after school. The sessions were mandatory. The content of each ses-sion emphasized basic competencies in the uniform national Mandarin lan-guage curriculum.

TABLE 2REGRESSIONS OF STUDENT CHARACTERISTICS ON INDICATORS OF ATTRITION AND TREATMENT DUMMIES

Mean Attrition TreatmentTreatment ×Attrition

ð1Þ ð2Þ ð3Þ ð4ÞStudent-level characteristics:

Baseline Mandarin scoreðunits of SDÞa .00 ð1.00Þ .33*** ð.12Þ .00 ð.16Þ .23 ð.18Þ

Baseline math score ðunits of SDÞa .00 ð1.00Þ .27 ð.15Þ .11 ð.15Þ .26 ð.22ÞMale ð1 5 yes; 0 5 noÞ .49 ð.50Þ .03 ð.06Þ .03 ð.03Þ .05 ð.08ÞOnly child ð1 5 yes; 0 5 noÞ .13 ð.34Þ .01 ð.03Þ .01 ð.02Þ .00 ð.05ÞFather illiterate ð1 5 yes; 0 5 noÞ .20 ð.40Þ .00 ð.06Þ .02 ð.03Þ .06 ð.07ÞMother illiterate ð1 5 yes; 0 5 noÞ .56 ð.50Þ .12*** ð.04Þ .03 ð.04Þ .01 ð.07ÞFamily off-farm .07 ð.26Þ .06 ð.03Þ .00 ð.02Þ .02 ð.05ÞLive with both parents .56 ð.50Þ .13*** ð.05Þ .04 ð.05Þ .12 ð.06ÞEver used a computer

ð1 5 yes; 0 5 noÞ .35 ð.48Þ .09 ð.06Þ .06 ð.07Þ .06 ð.09ÞAccess to other modern

technologiesb .47 ð.24Þ .00 ð.03Þ .01 ð.03Þ .04 ð.04ÞTibetan .21 ð.41Þ .14*** ð.05Þ .02 ð.10Þ .05 ð.07ÞTu .12 ð.32Þ .02 ð.03Þ .02 ð.06Þ .03 ð.05ÞHui .33 ð.47Þ .23*** ð.05Þ .04 ð.12Þ .01 ð.08ÞSalar .20 ð.40Þ .02 ð.05Þ .07 ð.12Þ .07 ð.07Þ

School-level characteristics:Number of students 229.82 ð105.7Þ 20.91 ð23.97Þ −50.48 ð31.67Þ −26.75 ð26.04ÞTeacher-student ratio .06 ð.01Þ .00 ð.00Þ .00 ð.00Þ .00 ð.00ÞProportion of female teachers .53 ð.19Þ .01 ð.03Þ .04 ð.05Þ .09 ð.05ÞSchool facility indexc .57 ð.17Þ .01 ð.05Þ .00 ð.05Þ .01 ð.05ÞProportion of students receiving

poverty subsidy .40 ð.27Þ .02 ð.09Þ .10 ð.08Þ .06 ð.10ÞCounty Huzhu .39 ð.49Þ .28*** ð.07Þ .06 ð.14Þ .00 ð.10ÞCounty Hualong .21 ð.41Þ .01 ð.06Þ .08 ð.12Þ .03 ð.08ÞCounty Xunhua .41 ð.49Þ .28*** ð.08Þ .02 ð.14Þ .03 ð.10Þ

Note. Robust standard errors clustered at the school level in parentheses. Each column is a regression ofthe row variable on the column variables. N 5 1,889.a Score on the standardized math/Mandarin test that was given to all students in the sample before thecomputer assisted learning ðCALÞ program.b Mean value of a set of 0/1 dummy variables including whether the student has used cell phone, Internet,game console, CAL software, and videos for learning assistance.c Mean value of a set of 0/1 dummy variables indicating the availability ð05 unavailable; 15 availableÞ ofsome school facilities, including enclosure, playground, reading room, library, running water, hot water,and clinic.*** Significant at 1%.

Lai et al. 761

During each session, two students formed a team to share one computerand played Mandarin games designed to help students review and practice thebasic Mandarin material that was being taught in their regular school Man-darin classes during the same week. In a typical session, the students firstwatched an animated video that reviewed the material that they learned duringthat particular week during their regular Mandarin class sessions. The studentswere able to playMandarin games to practice the knowledge and skills ðinclud-ing vocabulary, reading comprehension, and grammarÞ introduced in class. Ifa student had a Mandarin-related question, he or she was encouraged to dis-cuss it with his or her teammates. The students were not supposed to consultthe other teams or the teacher-supervisor. According to our protocol, the teach-ers were only allowed to help students with scheduling, computer hardwareissues, and software operations. According to our in-class observations, thesessions, in fact, were so intense that the attention of the students was fully onthe computer, and there was little communication among the groups or be-tween any of the groups and the teacher-supervisor.To facilitate the implementation, we then designed a detailed CAL curric-

ulum and implementation protocol. The protocol was targeted at the teacher-supervisors who were charged with implementing the CAL program in eachschool. One of the most important jobs of the teacher-supervisor was to makesure the CAL sessions were proceeding on a pace that matched the pace in thestudents’ regular Mandarin classes. To avoid confounding the effect of the CALintervention with any influence of additional teaching inputs to the students,none of the teacher-supervisors were Mandarin or math teachers or homeroomteachers of the third-grade students. An implementation protocol was pre-sented in a manual, which was given to the teacher-supervisor as a bound,printed booklet that contained detailed instructions.To ensure that the protocol would be properly implemented, we requested

that each school assign two teachers to supervise all of the CAL sessionsaccording to the protocol. Because this work was clearly beyond the scope oftheir normal classroom duties, we compensated the teacher-supervisors witha monthly stipend of ¥100 ðapproximately US$15Þ. This amount was equalto roughly 15% of the wage of a typical rural teacher. To prepare teacher-supervisors for their duties, before the spring 2011 semester started all teacher-supervisors of the 26 treatment schools attended a 2-day mandatory trainingthat was held at a central site.To further ensure that the teacher-supervisors ðand the students under their

supervisionÞ strictly followed the protocol, we recruited volunteers from uni-versities inHaidong Prefecture and directed them to visit the treatment schools


during the implementation of CAL.2 None of the visits were announced to theschools in advance. During the visits, the volunteers were instructed to attendthe CAL sessions and observe whether the protocol was being strictly imple-mented. They also were instructed to avoid all unnecessary interactions withstudents and teachers so that they would not interrupt the sessions or pro-vide additional assistance to CAL session management, as such interactionsmight confound the program effect. The research team also provided technicalsupport and free computer repairs and maintenance for the entire semester.According to the notes of the volunteer monitors, almost all schools closely

followed the CAL program instructions during implementation. During theend line survey, the interviewers also asked the students how often they had theCAL sessions ðmore than 99% said they had the sessions regularlyÞ, whetherany of the CAL sessions were taken up for other classes ðnone of the studentsreported that the CAL sessions substituted for other classesÞ, and whether theteacher-supervisors taught them any subject content during the CAL sessionsðaccording to the survey only 0.01% said that the teacher-supervisors occa-sionally offered them clues on how to solve problems during the CAL sessionswhen the students asked them for helpÞ. Therefore, we found few violations ofour rules for CAL implementation according to the student responses. Ac-cording to records kept by teacher-supervisors, there were almost no cancel-lations of CAL sessions, except for a very few due to irregular events ðe.g.,sudden electricity shutdownÞ. It should be noted, however, that whenever acancellation did occur, all schools arranged make-up sessions so that the CALsessions were managed according to our schedule.

CAL Control Group ðThird-Grade Students in the 26 Control SchoolsÞThird-grade students in the 31 control schools constituted the CAL controlgroup. Students in the control group did not receive any CAL intervention.To avoid any type of spillover effects of the CAL intervention, the principals,teachers, and students ðand their parentsÞ of the control schools were not in-formed of the CAL project. The research team did not visit the control schools

2 It is true that we ðthe outside research teamÞ visited the schools and gave baseline surveys andtests. However, these were done in both treatment and control schools. Apart from this, therewas almost no interaction between the research team and the students. In one or two sessionsstudents from local colleges visited the schools. However, they were there for only short periodsof time and had almost no interaction with the students or teachers. In these visits it was unclearthat they were really concerned fully with the CAL program. Hence, it is unlikely that any sig-nificant part of the observed treatment effect was from a Hawthorne effect. We took manyprecautions to ensure that any scope for Hawthorne effects was minimal.

Lai et al. 763

except for during the baseline and final evaluation surveys. The students in thecontrol group took their regular Mandarin classes at school as usual.

Data CollectionThe research group conducted two rounds of surveys in the 57 control andtreatment schools. The first-round survey was a baseline survey conductedwith all third graders in the 57 schools in late December 2010, right before theclosing of the fall semester and before any implementation of the CAL pro-gram had begun. The second-round survey was a final evaluation survey con-ducted at the end of the program in late June, a time that coincided with theend of the spring semester of 2011.In each round, the enumeration team visited each school and conducted a

two-part survey. In the first part, students were given a standardized Manda-rin test and a standardized math test.3 The Mandarin test included 40–45questions ðtests in different rounds included slightly different numbers ofquestionsÞ. The math test included 25 questions. Students were required tofinish tests in each subject in 25 minutes. All students took the Mandarin testbefore taking the math test. Our enumeration team strictly enforced timelimits and proctored the examinations.We use the scores of the students on theMandarin and math tests as our measures of student academic performance.In the second part of the survey, enumerators collected data on the char-

acteristics of students and their families. From this part of the survey we areable to create demographic and socioeconomic variables. The data set includesmeasures of whether the students were male, their ethnic group ðHan, Hui,Salar, Tu, and TibetanÞ, county, whether they were the only child in the fam-ily, whether their father was illiterate ðfather illiterateÞ, whether their motherwas illiterate ðmother illiterateÞ, whether their parents were still farmers andworked on the family farm or whether their parents worked off-farm ðfamilyoff-farmÞ, and whether students had ever used a computer or had ever had ac-cess to other modern technologies. To create indicators of parental care, dur-ing the survey the students were also asked whether their parents had migratedto some other location outside of his or her hometown or whether their par-ents stayed at home for most of the time during the semester ðlive with bothparentsÞ.In the second part of the survey, students were also asked to answer ques-

tions that could help us measure their noncognitive traits. To create indicatorsfor students’ attitudes toward schooling ðlike schoolÞ, the students were asked

3 Personnel from the Center for Examination of Beijing helped us pick questions for the testsfrom official examination books and exercise books.


to rate their attitude toward school on a 0–100 scale, where 0 indicates “ex-tremely hate school” and 100 indicates “extremely enjoy school.” The indi-cators of the self-confidence and the self-efficacy of studying Mandarin werecreated from the responses of students to a 10-item psychological scale forthe self-confidence and a seven-item psychological scale for the self-efficacy ofstudying Mandarin.4

Statistical MethodWe used ordinary least squares ðOLSÞ regression analysis ðboth with and with-out control variablesÞ to estimate how the academic and noncognitive out-comes changed in the treatment group relative to the control group. Our ba-sic OLS analysis regressed the end line outcome variables ði.e., postprogramoutcome valueÞ on the value of outcome variables at baseline and a dummyvariable of the treatment ðCAL interventionÞ status. We then included in thebasic model a set of variables to control for some systematic differences be-tween the treatment and control groups and improve precision. In all regres-sions, we accounted for the clustered nature of our sample by constructingHuber-White standard errors corrected for school-level clustering. The modelsare presented in order of increasing comprehensiveness.First, the basic model is

yis 5 a1 b � treatment s 1 v � y0is 1 εis; ð1Þ

where yis is the end line outcome variable for child i in school s, y0is measuresthe outcome variable of the same child at the baseline, treatments is a dummyvariable for a third-grade student attending a treatment school ðequal to onefor students in the treatment group and zero otherwiseÞ, and εis is a randomdisturbance term clustered at the school level.We used several variables to measure the student academic and noncog-

nitive outcomes ðyisÞ. The primary outcome variable of our analysis is thestudent academic outcome, measured by the student standardized Mandarintest score. We also included the student standardized math test score as anadditional academic outcome measure. By doing so, we are able to examinewhether there are any positive or negative spillovers of the CAL interventionto student academic performance in math, the other major subject in China’selementary schools apart fromMandarin. Importantly, in addition to variables

4 To measure the self-efficacy of studying Mandarin, a professor in psychometrics and mea-surement at Beijing Normal University helped us choose 12 indicators of math attitudes used inthe Trends in International Mathematics and Science Study, 2003, and developed a seven-itemscale of self-efficacy for the study of Mandarin that is appropriate to use under the context ofelementary schools in China.

Lai et al. 765

measuring academic outcomes, we included three noncognitive outcome var-iables, namely, like school, self-confidence, and self-efficacy in studying Man-darin.By construction, the coefficient of the dummy variable treatments, b, mea-

sures the difference in the outcome variable between the treatment and con-trol groups conditional on the baseline value of the outcome variable. In otherwords, b measures, holding the baseline value of the outcome variable con-stant, how well the treatment group does in terms of the outcome variableduring the program period relative to the control group.In order to improve the efficiency of the estimation, we built on the model

in equation ð1Þ by including a set of control variables:yis 5 a1 b � treatments 1 v � y0is 1 Xisg1 εis; ð2Þ

where all the variables and parameters are the same as those in equation ð1Þ,except that we added a set of control variables. Specifically, apart from y0is, thepreprogram outcome value for student i in school s, we controlled for Xis, avector of additional control variables. The variables in Xis are student and fam-ily characteristics ðmale, ethnic group ½Han, Tibetan, Tu, Hui, and Salar�, onlychild, father illiterate, mother illiterate, family off-farm, live with both par-ents, ever used a computer, and access to other modern technologyÞ. By includ-ing y0is and Xis as control variables, b in equation ð2Þ is an unbiased, efficientestimate of the CAL treatment effect.

ResultsTable 3 reports the estimated treatment effects of the CAL program on studentstandardized Mandarin and math test scores, with each column reporting es-timated coefficients from one regression of the outcome variable on explana-tory variables in the corresponding rows. Column 1 reports the estimatedcoefficients from regressions of the standardized Mandarin scores on the treat-ment dummy variable and student baseline Mandarin test scores. Column 2reports the results of the same regression but includes a set of control variablesðas shown in the corresponding rowsÞ. Columns 3 and 4 follow the samemodels as columns 1 and 2, respectively, except the subject is math rather thanMandarin. The top row of this table reports the estimated treatment effects ofeach model.The data show that grade 3 students in the treatment group improved sig-

nificantly more in their academic performance in Mandarin than students inthe control group ðtable 3, cols. 1 and 2Þ. Using the sample including allgrade 3 students in the 57 sample schools, the estimated CAL treatment effecton Mandarin test scores is equal to 0.14 standard deviations and is significant


at the 10% level using the basic model ðeq. ½1�; table 3, col. 1Þ. It increases to0.20 and becomes more significant ðat the 1% levelÞ using the model control-ling for the various student and school characteristics ðeq. ½2�; table 3, col. 2Þ.

Spillovers in the Student Math Test ScoresImpressively, the impacts of CAL were not only on the student Mandarin testscores—following the subject matter that was the focus of CAL—but also onthe student math test scores ðtable 3, cols. 3 and 4Þ. Specifically, compared totheir counterparts in the control schools, grade 3 students in the treatmentschools improved their standardmath test scores 0.22–0.23 standard deviationsmore over the program period. These results suggest that CAL intervention didnot improve the performance of students in Mandarin at the expense of theirperformance in math. Instead, the academic benefits from the CAL interven-tion spilled over to mathematics as well.Although the measured effect on math scores is larger than the measured

effect on language scores, it is important to note that our findings should not

TABLE 3ORDINARY LEAST SQUARES ESTIMATORS OF THE IMPACTS OF CAL PROGRAM ON STUDENT ACADEMIC OUTCOMES

Mandarin Math

ð1Þ ð2Þ ð3Þ ð4ÞTreatment ð1 5 treatment group; 0 5 control groupÞ .14* .20*** .23** .22***

ð.07Þ ð.07Þ ð.09Þ ð.07ÞControl variable:

Baseline Mandarin score ðunits of SDÞa .64*** .49*** Yesð.03Þ ð.03Þ

Baseline math score ðunits of SDÞa Yes .61*** .50***ð.02Þ ð.03Þ

Student characteristicsb Yes YesSchool characteristicsc Yes Yes

R2 .48 .53 .39 .44

Source. Authors’ survey.Note. Dependent variable: standardized post-CAL ðcomputer assisted learningÞ test score. Robust stan-dard errors clustered at the school level in parentheses. Each column reports the results of one regressionof the student final standardized Mandarin or math test scores over the program period on the row vari-ables. N 5 1,717.a Score on the standardized math/Mandarin test that was given to all students in the sample before theCAL program.b Includes male ð1 5 yes; no 5 0Þ, only child ð1 5 yes; no 5 0Þ, father illiterate ð1 5 yes; no 5 0Þ, motherilliterate ð15 yes; no5 0Þ, family off-farm ð1 5 yes; no5 0Þ, live with both parents ð1 5 yes; no5 0Þ, everused a computer ð15 yes; no5 0Þ, access to other modern technologies, and ethnic groups ðHan, Tibetan,Tu, Hui, and SalarÞ.c Includes number of students, teacher-student ratio, proportion of female teachers, school facility indexðas described in table 1Þ, and proportion of students receiving poverty subsidy.* Significant at 10%.** Significant at 5%.*** Significant at 1%.

Lai et al. 767

be interpreted to mean that the program helped students to learn more maththan Mandarin. The main reason for this is that, in general, these two subjectsalmost certainly have different levels of sensitivity to program intervention. Inother words, in general, an increase of 0.1 standard deviations in math scoresfrom some program intervention is typically thought to be easier to achievethan an increase of 0.1 standard deviations in Mandarin scores. The literaturein education supports this view ðe.g., Klein et al. 2000Þ. Specifically, Richð2013Þ has documented that student math performance is often more sen-sitive to external factors such as teacher effect and educational interventionsthan student performance in language.There might be several reasons for this positive spillover effect of the CAL

intervention to student math performance. First, the CAL intervention mighthave increased the general learning ability of the students, which helped im-prove their performance in other subjects including math. Second, the game-based CAL software made learning a more fun and engaging process and thusstimulated the student interest in learning Mandarin as well as other subjectsðincluding mathÞ.More importantly, as discussed above, it may be that the remedial tutoring

that improved the Mandarin skills of children who are not proficient in theLWC had a positive effect on other subjects that were being taught using theLWC. Specifically, all school subjects were taught in Mandarin, and the text-books were written inMandarin. Therefore, improvedMandarin language skillsvia CAL might also have helped the students better understand course materialsin math. According to the national uniform math curriculum, Chinese primaryschools begin aggressively introducing arithmetic word problems in grade 3.Indeed, the grade 3 math textbooks that were being used by the students inour sample schools contained tens of thousands of words and more than 1,000unique Chinese characters. Reflecting this, the word count of the math testof the final evaluation is more than 1,300 words. Therefore, it is entirelyplausible that as a result of improved reading comprehension ability via CALintervention, students in the treatment group outperformed those in the con-trol group.He et al. ð2008Þ also found that an Indian English Education Curriculum

implemented via teacher training improved student math and English scoresrather than just their English scores. They claimed that this might be due tothe fact that teachers implemented the interventions more efficiently and,thus, had more time available for other subjects. This is unlikely to be the casein our study, since the teacher-supervisors did not have the autonomy tosubstitute the time of CAL sessions for other subjects. The teacher-supervisorsin our CAL experiment also did not teach the major subjects to the students in


our sample ðand so they really would have been unable to provide extra tu-toringÞ. As discussed above, survey data and interviews with students at theend of the program also confirmed that the CAL sessions were not used forother purposes ðincluding classes in other subjectsÞ. Moreover, for the schoolsin our sample, there were almost no remedial tutoring sessions offered afterschool.Of course, we cannot completely exclude the possibility that the im-

provement in the math performance ðand also in the Mandarin subjectÞ of thestudents was the result of some other factors instead of due to improved lan-guage skills. For example, it is possible that the program might make studentsmore interested in learning all subjects, thus improving their performance inmath. Moreover, having an extra teacher ðin this case, the teacher-supervisorof the CAL sessionsÞ might also have some effect on student performance.However, as mentioned above, the CAL program was carefully designed andimplemented to minimize interactions between the teacher-supervisors andstudents. On the basis of our data, we believe these efforts were effective. Dur-ing both the baseline and end line surveys, we asked students about their in-terest in studying ði.e., the indicator of “like school”Þ and found no signifi-cant increase in the treatment group compared to the control group ðtable 4,col. 3Þ. As for the extra teacher effect, there also is little reason to believe thatthe schools intentionally assigned high-quality teachers as the CAL session su-pervisors. Moreover, in the literature, according to our extensive search, thereis no conclusive empirical evidence that having an extra teacher ðregardless ofqualityÞ could systematically improve student performance. For these reasonswe argue that while we cannot rule out the possibility of confounding influ-ences mentioned above, given the information from our data and in-the-fieldobservations, we believe such effects, if there are any, only had a narrow influ-ence on the validity of our empirical results.

Effects of the CAL Intervention on Student Nonacademic OutcomesTable 4 reports the estimated treatment effects of the CAL program on studentnonacademic outcomes, with each column reporting estimated coefficientsfrom one regression of the outcome variable ðas the column titleÞ on explan-atory variables in the corresponding rows. Columns 1–3 report the estimatedcoefficients from regressions of the student scores on self-efficacy in Mandarinstudy, self-confidence, and how much students like schooling, respectively,on the treatment dummy variable. All regressions controlled for the studentbaseline values of the nonacademic outcomes as well as a set of control co-variates including student and school characteristics. The top row of this tablereports the estimated CAL treatment effects for each nonacademic outcome.

Lai et al. 769

Our results show that the CAL intervention had positive effects on thestudent nonacademic outcomes, although they are not statistically significantin most of the cases ðtable 4Þ. Compared to students in the control group, thestudents in the treatment group like school more and had higher levels of self-efficacy of studying Mandarin and self-confidence. In particular, students inthe treatment group had significantly higher levels of self-efficacy of studyingMandarin than those in the control group ðcol. 1Þ, yet the differences betweenthe treatment and control groups in the other two indicators were insignifi-cant ðcols. 2 and 3Þ.

Heterogeneous Effects of the CAL Intervention on Student Academic and NonacademicOutcomes for Students with Different Academic BackgroundsApart from the average impact of the CAL program on student academic andnonacademic performance, it is also important to explore whether the CALprogram has different impacts for different students. In particular, as this CALlanguage program is remedial in nature ðhelping students with difficulties inLWCÞ, we are interested in understanding whether the CAL intervention ismore effective in the case of students with lower academic backgrounds. To dothis, we use equation ð2Þ but include interaction terms between the treatment

TABLE 4ORDINARY LEAST SQUARES ESTIMATORS OF THE IMPACTS OF CAL

PROGRAM ON STUDENT NONACADEMIC OUTCOMES

Self-Efficacy ofStudying Mandarin Self-Confidence Like School

ð1Þ ð2Þ ð3ÞTreatment ð1 5 treatment group;

0 5 control groupÞ .07* .02 .37ð.04Þ ð.04Þ ð1.46Þ

Baseline values of the nonacademicoutcome indicator and othercontrol variables ðstudent andschool characteristicsÞ Yes Yes

Yes

R2 .15 .12 .05

Source. Authors’ survey.Note. Dependent variables: indicators of nonacademic outcomes. Baseline math/Mandarinscore is the score on the standardized math/Mandarin test that was given to all students in thesample before the computer assisted learning ðCALÞ program. Other control variables arestudent characteristics including male ð1 5 yes; no 5 0Þ, only child ð1 5 yes; no 5 0Þ, fatherilliterate ð1 5 yes; no 5 0Þ, mother illiterate ð1 5 yes; no 5 0Þ, family off-farm ð1 5 yes; no 50Þ, live with both parents ð15 yes; no5 0Þ, ever used a computer ð15 yes; no5 0Þ, access toother modern technologies, and ethnic groups ðHan, Tibetan, Tu, Hui, and SalarÞ and schoolcharacteristics including number of students, teacher-student ratio, proportion of femaleteachers, school facility index ðas described in table 1Þ, and proportion of students receivingpoverty subsidy. Each column reports the results of one regression of the student nonaca-demic outcome on the corresponding row variables. Robust standard errors clustered at theschool level in parentheses. N 5 1,715.* Significant at 10%.


variable and student academic background ðwhich is measured using studentbaseline Mandarin and math scoresÞ.The main purpose of the heterogeneity analysis is to detect the existence of

heterogeneous program effects. For this reason we believe that a linear func-tion form is enough for detecting the heterogeneous effect and describing itsgeneral pattern. All the same, we explored alternative specifications for het-erogeneity analysis. Specifically, we plot the change in the student Mandarinstandardized test scores over the program period for each group ðtreatmentgroup and control groupÞ versus the baseline Mandarin standardized testscores ðfig. 2Þ. According to our findings, the difference in the change in testscores over the program period between the treatment and control groupsdecreases monotonically as the baseline Mandarin test score goes up. There-fore, we conclude that the linear specification is a reasonable choice in our case.The results of the heterogeneous CAL treatment effects across student

academic background are summarized in table 5, with each column reportingestimated coefficients from one regression of the outcome variable ðas thecolumn titleÞ on explanatory variables in the corresponding rows. Column 1reports the estimated coefficients from regressions of the standardized studentMandarin scores on the treatment dummy variable, student baseline Man-darin test scores, and the interactions of these two variables. Column 2 reportsthe results of the same regression but uses student baseline math test scoresinstead of Mandarin test scores on the right-hand side. Columns 3 and 4follow the same models as columns 1 and 2, respectively, except we change theoutcome fromMandarin test scores to student self-efficacy of Mandarin study.Like table 4, all regressions controlled for a set of student and school char-acteristics. Table 5 reports how the estimated treatment effects of each modelvary across the distribution of the test scores. It also reports the q-value FDRðfalse discovery rate, or probability of falsely reject the null hypothesisÞ of eachtest of heterogeneous effect.The results show that the CAL intervention benefited low-performing

students more than it did high-performing students ðtable 5, interactionsÞ.Compared to the students in the control group, students in the treatmentgroup with a baseline Mandarin test score 1 standard deviation lower than themean level improved 0.16 standard deviations more in their Mandarin testscores over the program period ðsignificant at the 1% level; col. 1Þ. Comparedto the students in the control group, students in the treatment group with abaseline math test score 1 standard deviation lower than the mean level alsoimproved 0.09 standard deviations more in their Mandarin test scores over theprogram period ðsignificant at the 10% level; col. 2Þ. This finding indicatesthat CAL has been successful in achieving its goal of helping low-performing

Lai et al. 771

students catch up. We also tested heterogeneous effects of the CAL interven-tion on student math performance, but no significant patterns of heteroge-neous CAL intervention effects were found ðresults not included for brevityÞ.5Using the same estimation model we did with the student academic out-

come, we found that low-performing students benefited more than otherstudents from the CAL intervention not only in academic performance butalso in some nonacademic outcomes. Compared to the students in the controlgroup, students in the treatment group with lower-than-average baseline testscores in Mandarin and math improved significantly more in their levels of

5 The results of the heterogeneity analysis looking at the insignificant coefficient of the treat-ment × baseline test score on math performance should not be surprising. One possible ex-planation is that this is so because the improvement in math ðthat we are observing for the fullsampleÞ is a product of two things: the CAL program in Mandarin ðand the improvement inlanguage from the CAL programÞ and the beginning level of one’s Mandarin language abilityðwhich may be strongly correlated with the ability to translate improvement in Mandarin com-petency into improvement in math performanceÞ. Hence, although students with lower base-line standardized Mandarin test scores gained more in their Mandarin performance ðand thismay have contributed to higher math scoresÞ, it is also true that they began with lower levels ofMandarin comprehension ðand this may have limited the improvement in their math scoresÞ.Therefore, it is plausible that even with the relatively higher gains inMandarin language skills, thelower-performing students ðat the baselineÞ were unable to translate those into relatively highergains in math after the Mandarin-CAL treatment.

Figure 2. Change in student standardized Mandarin test score during the program period versus studentbaseline standardized Mandarin test score.


self-efficacy of studying Mandarin than other students in the treatment groupover the program period ðtable 5, cols. 3 and 4Þ.It is possible that while testing heterogeneous effects across a large number

of characteristics ð11 student characteristicsÞ, we might by chance find thatthe academic background characteristics are significant. To ensure integrity ofour analysis, we conducted an additional FDR calculation to examine whetherthe observed heterogeneous treatment effects across student academic back-grounds were genuine. For controlling the FDR, we used the sharpened two-stage q-values thatwere introduced inBenjamini,Krieger, andYekutieli ð2006Þ.We find that at least for the heterogeneous effect of CAL treatment on studentMandarin test scores across student baseline Mandarin test scores and the het-erogeneous effect of CAL treatment on student self-efficacy of studying Man-darin across student baseline math test scores, the q-values ðthe smallest FDR,or the probability of falsely rejecting the null hypothesisÞwas only 0.06 ðtable 5,

TABLE 5ORDINARY LEAST SQUARES ESTIMATORS OF THE HETEROGENEOUS PROGRAM EFFECT

ON STUDENTS WITH DIFFERENT LEVELS OF ACADEMIC PERFORMANCE

StandardizedPost-CAL Mandarin

Test Score

Self-Efficacyof StudyingMandarin

ð1Þ ð2Þ ð3Þ ð4ÞTreatment ð1 5 treatment group; 0 5 control groupÞ .21*** .19*** .08* .07*

ð.07Þ ð.06Þ ð.04Þ ð.04ÞInteractions:

Treatment × baseline Mandarin score ðunits of SDÞa .16*** .06**ð.05Þ ð.03Þ

Treatment × baseline math score ðunits of SDÞa .09* .09***ð.04Þ ð.03Þ

q-value ðfalse discovery rateÞ .06 .18 .25 .06Control variables Yes Yes Yes YesR2 .24 .23 .23 .23

Source. Authors’ survey.Note. Control variables include the baseline math/Mandarin score, student characteristics including maleð15 yes; no5 0Þ, only child ð15 yes; no5 0Þ, father illiterate ð15 yes; no5 0Þ, mother illiterate ð15 yes;no 5 0Þ, family off-farm ð1 5 yes; no 5 0Þ, live with both parents ð1 5 yes; no 5 0Þ, ever used a computerð1 5 yes; no 5 0Þ, access to other modern technologies, and ethnic groups ðHan, Tibetan, Tu, Hui, andSalarÞ and school characteristics including number of students, teacher-student ratio, proportion of femaleteachers, school facility index ðas described in table 1Þ, and proportion of students receiving povertysubsidy. Each column reports the coefficients of the treatment dummy variable and the interaction of thetreatment dummy variable and the corresponding student characteristics from one regression. Eachregression regresses the dependent variable on the treatment dummy variable and the interaction of thetreatment dummy variable and the corresponding student characteristics in that column, controlling for allof the control variables. Robust standard errors clustered at the school level in parentheses. N 5 1,717.a Score on the standardized math/Mandarin test that was given to all students in the sample before thecomputer assisted learning ðCALÞ program.* Significant at 10%.** Significant at 5%.*** Significant at 1%.

Lai et al. 773

cols. 1 and 4Þ. In other words, the heterogeneous effects were significant whencontrolling FDR5 0.06. This indicates that the observed heterogeneous CALtreatment effects across student academic backgrounds ðmeasured by the stu-dent baseline Mandarin test scores and math test scoresÞ are unlikely to have justhappened by chance.

ConclusionOn the basis of a randomized experiment using a computer assisted learningðCALÞ program, we examined the impact of the language of wider commu-nication ðLWCÞ learning on overall academic performance ðas well as otheroutcomesÞ. The sample of students in our study mostly have limited compe-tency in the LWC. The study involves more than 1,700 third-grade students,age 9–11, in 57 public schools in poor rural minority areas in Haidong Pre-fecture, Qinghai. To evaluate the effectiveness of the LWC learning program,we randomly chose 26 schools from the sample as treatment schools. As partof the treatment the third-grade students attending these schools received theMandarin after-school remedial program. The remaining 31 schools served ascontrol schools. The program was tailored to the regular school Mandarin cur-riculum and was remedial in nature.Our results indicate that improving the LWC skills of ethnic minority

students has significant positive effects on student language and math testscores as well as on nonacademic outcomes. Two 40-minute Mandarin ses-sions per week increased student standardized Mandarin scores by 0.14–0.20 standard deviations. In general, low-performing students benefited morefrom the program. Perhaps more importantly, the intervention that was fo-cused fully on teaching the LWC also had a significant impact on the math testscores of the students, with the magnitudes of the effects varying from 0.22 to0.23 standard deviations. The intervention appeared to have positive impactson the nonacademic outcomes of the students, although impacts were notstatistically significant in most cases. So, in short, the study highlights the im-portance of efforts in improving student LWC skills to the academic perfor-mance of students with limited LWC competency. Specifically, our study pro-vides rigorous evidence that a CAL program is a quick and effective means toimprove student LWC skills where educational resources are scarce and thatimproved LWC competency results in better academic performance in othersubjects ðin this case, mathÞ.The findings of our study are important for several reasons. First, the re-

sults are important because ours is one of the first studies in the context ofa developing country to show this relationship ðthat LWC skills contributeto overall academic performanceÞ. Second, our findings can be interpreted as


being truly causal—since the study was done in the context of large-scale ran-domized controlled trial. Third, our results show that the positive relation-ship between efforts in improving LWC skills and overall academic perfor-mance holds when the LWC is not English; indeed this is a study of the effectof teaching Mandarin on math test scores, which is outside the usual envi-ronment ðteaching Spanish-speaking Latino children English in the UnitedStatesÞ in which these studies have been conducted in the past. Overall, ourstudy shows that a CAL-based LWC skills program has the potential to be arelatively quick and effective remedy to the underperformance of disadvan-taged students in developing countries with limited LWC competency.

ReferencesAlba, R. D., J. Handl, and W. Müller. 1994. “Ethnische Ungleichheit im DeutschenBildungssystem.” KölnerZeitschrift für Soziologie und Sozialpsychologie 46:209–37.

Aud, S.,W.Hussar, G.Kena, K. Bianco, L. Frohlich, J. Kemp, andK.Tahan. 2011.TheCondition of Education 2011. NCES 2011-033. Washington, DC: National Centerfor Education Statistics.

August, D., and T. Shanahan, eds. 2006. Developing Literacy in Second-LanguageLearners. Mahwah, NJ: Erlbaum.

Bamgbose, A. 1984. “Mother-Tongue Medium and Scholastic Attainment in Nige-ria.” Prospects 14, no. 1:87–93.

Banerjee, A., S. Cole, E. Duflo, and L. Linden. 2007. “Remedying Education: Evi-dence from Two Randomized Experiments in India.” Quarterly Journal of Econom-ics 122, no. 3:1235–64.

Barrow, L., L. Markman, and C. Rouse. 2008. “Technology’s Edge: The EducationalBenefits of Computer-Aided Instruction.” NBERWorking Paper no. 14240, Na-tional Bureau of Economic Research, Cambridge, MA.

Benjamini, Y., A. M. Krieger, and D. Yekutieli. 2006. “Adaptive Linear Step-UpProcedures That Control the False Discovery Rate.” Biometrika 93, no. 3:491–507.

Cheung, A., and E. Slavin. 2005. “Effective Reading Programs for English LanguageLearners and Other Language-Minority Students.” Bilingual Research Journal 29,no. 2:241–67.

Genesee, F., K. Lindholm-Leary, K. Saunders, and D. Christian. 2006. EducatingEnglish Language Learners: A Synthesis of Research Evidence. New York: CambridgeUniversity Press.

Glewwe, P., and M. Kremer. 2006. “Schools, Teachers, and Education Outcomesin Developing Countries.” In Handbook on the Economics of Education, ed. E.Hanushek and F. Welch, 945–1017. New York: Elsevier.

He, F., L. Linden, and M. MacLeod. 2008. “How to Teach English in India: Testingthe Relative Productivity of Instruction Methods within the Pratham English Lan-guage Education Program.” Unpublished manuscript, Department of Economicsand School of International and Public Affairs, Columbia University.

Lai et al. 775

IDRC ðInternational Development Research CenterÞ. 1997. Languages of Instruction,Policy Implications for Education in Africa. Ottawa: International DevelopmentResearch Center.

Klein, S., L. Hamilton, D. McCaffrey, and B. Stecher. 2000. “What Do Test Scores inTexas Tell Us?” RAND Issue Paper no. 202, RAND Corporation, Santa Monica,CA.

Lai, F., R. Luo, L. Zhang, X. Huang, and S. Rozelle. 2014. “Does Computer-AssistedLearning Improve Learning Outcomes? Evidence from a Randomized Experimentin Migrant Schools in Beijing.” REAP Working Paper no. 228, Rural EducationAction Program, Stanford, CA.

Lai, F., L. Zhang, X. Hu, Q. Qu, Y. Shi, Y. Qiao, M. Boswell, and S. Rozelle. 2013.“Computer Assisted Learning as Extracurricular Tutor? Evidence from a Random-ised Experiment in Rural Boarding Schools in Shaanxi.” Journal of Development Ef-fectiveness 5, no. 2:208–31.

Master, B., S. Loeb, and J. Wyckoff. 2013. “Learning That Lasts: Unpacking Vari-ation in Teachers’ Effects on Students’ Long-Term Knowledge.” Unpublished man-uscript, Center for Education Policy Analysis, Stanford University.

Mo, D., L. Zhang, R. Luo, Q. Qu, W. Huang, J. Wang, Y. Qiao, M. Boswell, and S.Rozelle. 2014. “Integrating Computer Assisted Learning into a Regular Curricu-lum: Evidence from a Randomized Experiment in Rural Schools in Shaanxi.”Journal of Development Effectiveness 6, no. 3:300–323.

Planty, M., W. Hussar, T. Snyder, S. Provasnik, G. Kena, R. Dinkes, A. KewalRamani, and J. Kemp. 2008. The Condition of Education 2008. Washington, DC:National Center for Education Statistics.

Rich, M. 2013. “In Raising Scores, 123 Is Easier than ABC.”New York Times, May 29.Slavin, R., C. Lake, B. Chambers, A. Cheung, and S. Davis. 2009. “Effective ReadingPrograms for the Elementary Grades: A Best-Evidence Synthesis.” Review of Ed-ucational Research. 79, no. 4:1391–1466.

Slavin, R., N. Madden, M. Calderón, A. Chamberlain, and M. Hennessy. 2011.“Reading and Language Outcomes of a Multiyear Randomized Evaluation ofTransitional Bilingual Education.” Educational Evaluation and Policy Analysis 33,no. 1:47–58.

World Bank. 2001. China: Challenges of Secondary Education. 2nd ed. Washington,DC: World Bank.

———. 2004. World Development Report 2004: Making Services Work for the Poor.New York: Oxford University Press.

Yang, Y., H. Wang, L. Zhang, S. Sylvia, R. Luo, Y. Shi, W. Wang, and S. Rozelle.2013. “The Han-Minority Achievement Gap, Language and Returns to Schools inRural China.” REAP Working Paper no. 258, Rural Education Action Program,Stanford, CA.

Yang, Y., L. Zhang, J. Zeng, X. Pang, F. Lai, and S. Rozelle. 2013. “Computers andthe Academic Performance of Elementary School-Aged Girls in China’s Poor Com-munities.” Computers and Education 60:335–46.

Yates, R. 1995. “Functional Literacy and the Language Question.” International Jour-nal of Educational Development 15, no. 4:437–47.


Copyright of Economic Development & Cultural Change is the property of University ofChicago Press and its content may not be copied or emailed to multiple sites or posted to alistserv without the copyright holder's express written permission. However, users may print,download, or email articles for individual use.

teaching the language of wider communication, minority ......teaching the language of wider...

Documents