AMERICAN INSTITUTES FOR RESEARCHAMERICAN INSTITUTES FOR RESEARCH

An Analysis of the Research and Impact of ICT in Education in Developing Country Contexts

Nitika Tolani-Brown, PhD

Meredith McCormac, MA

Roy Zimmermann, PhD

AMERICAN INSTITUTES FOR RESEARCH

BackgroundBackground

Within the education sector, the importance of ICT lies in its Within the education sector, the importance of ICT lies in its ability to provide increased access to knowledge and ability to provide increased access to knowledge and services that such tools provide marginalized or services that such tools provide marginalized or traditionally underserved populations. traditionally underserved populations.

There is a sense of positive impact among practitioners but There is a sense of positive impact among practitioners but no rigorous research to support “feeling”no rigorous research to support “feeling”

In the absence of rigorous experimental literature, it is easy In the absence of rigorous experimental literature, it is easy to persist in the notion that ICT is “a magic bullet that will to persist in the notion that ICT is “a magic bullet that will provide the answer to long standing educational provide the answer to long standing educational challenges”challenges”

This review is a first step to establishing recommendations This review is a first step to establishing recommendations and developing a research agenda for academicians, and developing a research agenda for academicians, practitioners and policymakers to estimate impacts of ICT practitioners and policymakers to estimate impacts of ICT on educational outcomes in the developing worldon educational outcomes in the developing world

AMERICAN INSTITUTES FOR RESEARCH

Scope Scope

Developing country contexts Developing country contexts Experimental, quasi-experimental and qualitative/case Experimental, quasi-experimental and qualitative/case

studiesstudies Educational outcomes for learners, excluding impacts Educational outcomes for learners, excluding impacts

on other beneficiarieson other beneficiaries• Clear consensus has not emerged on the direction Clear consensus has not emerged on the direction

or the magnitude of ICTs effects on studentsor the magnitude of ICTs effects on students Policymakers need to know more about the Policymakers need to know more about the

combinations of factors that enhance or mitigate the combinations of factors that enhance or mitigate the effects of ICTs on students in educational settings if effects of ICTs on students in educational settings if they are to make decisions based on evidence rather they are to make decisions based on evidence rather than intuition. than intuition.

AMERICAN INSTITUTES FOR RESEARCH

MethodologyMethodology

Literature ReviewLiterature Review

Qualitative Interviews with ExpertsQualitative Interviews with Experts• Contacted 40 experts, including researchers, Contacted 40 experts, including researchers,

academicians, policymakers and practitioners academicians, policymakers and practitioners in the US, Europe and developing countries in the US, Europe and developing countries (e.g. Zambia, Nepal and Afghanistan)(e.g. Zambia, Nepal and Afghanistan)

• 25 interviews conducted25 interviews conducted

AMERICAN INSTITUTES FOR RESEARCH

Qualitative LiteratureQualitative Literature

Small pool of purely qualitative evaluations within our Small pool of purely qualitative evaluations within our criteriacriteria

Our report highlights three evaluations that represent a Our report highlights three evaluations that represent a diverse set of approaches and methodological rigor within diverse set of approaches and methodological rigor within the qualitative arenathe qualitative arena

Methodologies included in-depth structured interviews or Methodologies included in-depth structured interviews or focus group discussions with program beneficiaries at the focus group discussions with program beneficiaries at the local and policy levels, classroom and school observations, local and policy levels, classroom and school observations, and literature reviewsand literature reviews

Case studies alone do not provide a rigorous enough base Case studies alone do not provide a rigorous enough base from which to draw concrete programmatic and policy from which to draw concrete programmatic and policy recommendations or impact estimatesrecommendations or impact estimates

Need for replication – a study’s findings may not be Need for replication – a study’s findings may not be relevant after a few yearsrelevant after a few years

AMERICAN INSTITUTES FOR RESEARCH

Experimental EvaluationsExperimental Evaluations

Very few experimental evaluations have been conducted on the Very few experimental evaluations have been conducted on the impact of ICT on student outcomesimpact of ICT on student outcomes

Experimental Evaluations are important:Experimental Evaluations are important:• Provide internal and external validity and generalizability Provide internal and external validity and generalizability RCTs can help identify differential effects across subgroups RCTs can help identify differential effects across subgroups

that might otherwise remain uncoveredthat might otherwise remain uncovered Boys versus girls, special needs students, socio-economic Boys versus girls, special needs students, socio-economic

status, etcstatus, etc Report highlights three key RCTs conducted in India on Report highlights three key RCTs conducted in India on

computer assisted learning programscomputer assisted learning programs Results suggest the importance of individualizing ICT Results suggest the importance of individualizing ICT

solutions and investigating how ICT solutions interact with solutions and investigating how ICT solutions interact with the learning environment and subsequently impact student the learning environment and subsequently impact student outcomesoutcomes

AMERICAN INSTITUTES FOR RESEARCH

Challenges to Evaluation in Challenges to Evaluation in Developing CountriesDeveloping Countries

Absence of standard framework for evaluation of Absence of standard framework for evaluation of ICT programs in educational settingsICT programs in educational settings

Limited local capacity for evaluationLimited local capacity for evaluation

Limited funding/scarce resourcesLimited funding/scarce resources

Lack of demand and ownership of evaluationLack of demand and ownership of evaluation

AMERICAN INSTITUTES FOR RESEARCH

Lessons Learned & Lessons Learned & RecommendationsRecommendations

Impacts of ICT are varied in literatureImpacts of ICT are varied in literature• Positive, negative and insignificant impacts on learning Positive, negative and insignificant impacts on learning

outcomes are reportedoutcomes are reported

Research suggests type of impact often Research suggests type of impact often contingent upon other variables, including contingent upon other variables, including teacher pedagogy, infrastructure, learning teacher pedagogy, infrastructure, learning environment, etcenvironment, etc

Additionally, design of many ICT interventions is Additionally, design of many ICT interventions is based upon perceptions and anecdotal evidencebased upon perceptions and anecdotal evidence

AMERICAN INSTITUTES FOR RESEARCH

Lessons Learned & Lessons Learned & Recommendations (cont’d)Recommendations (cont’d)

Design longitudinal studiesDesign longitudinal studies Look for differential rates of impact across Look for differential rates of impact across

subgroupssubgroups Evaluations conducted utilizing mixed methods Evaluations conducted utilizing mixed methods

could be highly usefulcould be highly useful Develop international standards for evaluation of Develop international standards for evaluation of

ICT’s impact on learningICT’s impact on learning Explore other research areas, such as effects of Explore other research areas, such as effects of

learning styles, teacher training, and otherslearning styles, teacher training, and others