twenty years of mall project implementation: a meta-analysis of learning outcomes
DESCRIPTION
Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes. CALICO 2014. Jack Burston Honorary Research Fellow Cyprus University of Technology. MALL Learning Outcomes: 1994-2012. Few statistically reliable measures of learning outcomes - PowerPoint PPT PresentationTRANSCRIPT
Twenty Years of MALL Project Implementation: A Meta-analysis of Learning Outcomes
Jack BurstonHonorary Research FellowCyprus University of Technology
CALICO 2014
MALL Learning Outcomes: 1994-2012
• Few statistically reliable measures of learning outcomes
• Over 40% of MALL publications unrelated to language learning applications
• Pervasive methodological shortcomings
MALL Research Problems
• Sample size and treatment duration
Figure 1: MALL Project Duration
MALL Research Problems
• Sample size and treatment duration
Figure 2: MALL Student Numbers
MALL Research Problems
Minimal requirements for statistical analysis:
• 4 weeks duration (20 class days)
• 10 experimental subjects
= 109 MALL studies
MALL Research Problems
• Lack of quantifiable data:
74 studies contain no quantitative learning outcome data
= 35 MALL studies with objective learning outcomes from implementations that lasted at least a month and involved ten or more experimental subjects
MALL Research Problems
• Research design shortcomings– Student usage of mobile devices not tracked (3) – Inadequate description of the control group (3)– Uncontrolled variables (4)– Confounding variables (3) – Inadequate statistical analysis (3)
= 19 valid research studies
VALID MALL Studies: General Characteristics
Figure 3: General Characteristics of MALL Studies
VALID MALL Studies: Duration of Study
Figure 4: Duration of Study
VALID MALL Studies: Group Size
Figure 5: Group Size
VALID MALL Studies: Year of Study
Figure 6: Year of Study
VALID MALL Studies: Research Findings
Vocabulary Acquisition: 11/19L2 English8/11 SMS/MMS on basic mobile phones3/11 PDAs and smartphones
VALID MALL Studies: Research Findings
5/11 = Positive MALL advantage 3 SMS (Iran) 2 MMS (Turkey, Taiwan)
3/11 SMS Mixed results 2 Delayed post-tests contradict
initial results (China, Iran) 1 No significant difference (Iran)
VALID MALL Studies: Research Findings
3/11 Non SMS/MMS projects2 PDA-based projects positive
results (Taiwan)1 Smartphone negative
results (Japan)
VALID MALL Studies: Research Findings
Reading Competency: 4/19 All PDA-based All positive MALL results
2 L1 Spanish (Chile) 2 L2 English (Taiwan)
VALID MALL Studies: Research Findings
Listening/Speaking Skills: 3/19 Mobile devices used as audio recording tools All positive MALL results
L2 English (Taiwan)L2 Indonesian (Australia)L2 English (Cyprus)
VALID MALL Studies: Research Findings
Writing Skills: 1/19 Unspecified mobile device L2 English (Taiwan) Positive results
VALID MALL Studies: Conclusion
• Only 35 MALL studies meet minimal conditions of duration and sample size
• Only 19 MALL studies provide statistically reliable outcome data
• 15 Unequivocally positive results (80%): reading, listening, speaking and writing
• 4 No significant difference: vocabulary
MALL Studies: Fundamental Problems
• Inadequate research design• Technocentricity
Thank you for your attention!
Any questions?
MALL Learning Outcomes: 1994-2012
Burston, J. (2014). 20 years of MALL project implementation: A meta-analysis of learning outcomes. ReCALL Journal, 27(1), xxx-xxx. (available online)
Burston, J. (2014). The reality of MALL project implementations: Still on the fringes. CALICO Journal, 31(1), 43-65.
Burston, J. (2013). Mobile-Assisted Language Learning: A selected annotated bibliography of implementation studies 1994-2012. Language Learning & Technology, Special volume on Mobile-Assisted Language Learning, 17(3), 157–224.
MALL Learning Outcomes: Bibliography