digital literacy preschoolers 6/26/2015 · digital literacy for preschoolers 6/26/2015 conference...
TRANSCRIPT
![Page 1: Digital Literacy Preschoolers 6/26/2015 · Digital Literacy for Preschoolers 6/26/2015 Conference at McGill University 4 *Prototype (BLTK) built in 1998 *ABRACADABRA in 2002 (usability](https://reader035.vdocument.in/reader035/viewer/2022070903/5f5e89e652f74428e240a887/html5/thumbnails/1.jpg)
Digital Literacy for Preschoolers 6/26/2015
Conference at McGill University 1
A Cluster Randomized Control Field Trial of the ABRACADABRA Web-based Reading Technology: Replication and Extension of Basic
Findings.
McGill University Digital Literacy for Preschoolers Conference June 2015
http://abralite.concordia.ca
*
Student Module: Instruction
Choose from skill type OR story Four Main Skills: Alphabetics, Fluency, Comprehension and Writing 32 Activities
Digital Stories: 17 Stories + 15 students’ stories
![Page 2: Digital Literacy Preschoolers 6/26/2015 · Digital Literacy for Preschoolers 6/26/2015 Conference at McGill University 4 *Prototype (BLTK) built in 1998 *ABRACADABRA in 2002 (usability](https://reader035.vdocument.in/reader035/viewer/2022070903/5f5e89e652f74428e240a887/html5/thumbnails/2.jpg)
Digital Literacy for Preschoolers 6/26/2015
Conference at McGill University 2
Built-in Scaffolding and Support
Models of strategies are consistent
Visual and audio support
Skill Development
Choose a specific skill: Letter sounds
Reading
Understanding the story or
Writing
Multiple Levels
![Page 3: Digital Literacy Preschoolers 6/26/2015 · Digital Literacy for Preschoolers 6/26/2015 Conference at McGill University 4 *Prototype (BLTK) built in 1998 *ABRACADABRA in 2002 (usability](https://reader035.vdocument.in/reader035/viewer/2022070903/5f5e89e652f74428e240a887/html5/thumbnails/3.jpg)
Digital Literacy for Preschoolers 6/26/2015
Conference at McGill University 3
Teacher Module
Teacher GuideSee also: http://grover.concordia.ca/abracadabra/resources
Stories
Printed Resources
Technical Resources (FAQs)
![Page 4: Digital Literacy Preschoolers 6/26/2015 · Digital Literacy for Preschoolers 6/26/2015 Conference at McGill University 4 *Prototype (BLTK) built in 1998 *ABRACADABRA in 2002 (usability](https://reader035.vdocument.in/reader035/viewer/2022070903/5f5e89e652f74428e240a887/html5/thumbnails/4.jpg)
Digital Literacy for Preschoolers 6/26/2015
Conference at McGill University 4
*Prototype (BLTK) built in 1998
*ABRACADABRA in 2002 (usability testing)
*Pilot Study 2004 (small effect sizes)
*Larger study 2005-2006 K/ 1st Grade (Savage et al., 2009)
*Pan-Canadian external trial RCT 2007- 2009 (Savage et al., 2013)
*Australian studies 2008-2010 (Wolgemuth et al 2011, 2013, 2014)
*ABRA-ePEARL Connection 2010- 2012
929496
98100102
104106
108110
Pre-test Post-test
Phoneme
Rime
Control
CTOPP blending standard score GRADE listening comprehension (stanine)
0
1
2
3
4
5
6
Pre-test Post-test
Phoneme
Rime
Control
8485
868788
8990
91
92
93
Post-test
Phoneme
Rime
Control
GRADE reading comprehension scores
0
5
10
15
20
Pre-test Post-test Follow up
Phoneme
Rime
Control
Woodcock-Johnson Fluency (raw scores)
*Foci: To learn how teachers use ABRA technology in their ELA lessons and to see if ABRA has an impact on students’literacy development.
*Pan-Canadian focus* Alberta, Ontario and Quebec participants
*Pre- and post-tests administered*Randomized controlled trial in 76 classrooms (36
experimental and 36 control)* Kindergarten, Grade 1 students* Over 1000 children participated* 10 – 12 weeks of intervention
*Significant beneficial effects on childrens’:
Letter/sound knowledge
Word reading
Phonological awareness
*A look at implementation suggests:
Teachers use ABRA as a resource to teach phonics (consistent with effect patterns above) only.
Table1.
ResearchonABRACADABRA:BestEvidenceonImpacts
Reading Skill k (# ofcomparisons)
AverageEffectSize
PercentileAdvantage
Alphabetics 21 +0.396 15.39Fluency 19 +0.187 7.42Comprehension 11 +0.340 13.31
Overall 51 +0.306 12.02
![Page 5: Digital Literacy Preschoolers 6/26/2015 · Digital Literacy for Preschoolers 6/26/2015 Conference at McGill University 4 *Prototype (BLTK) built in 1998 *ABRACADABRA in 2002 (usability](https://reader035.vdocument.in/reader035/viewer/2022070903/5f5e89e652f74428e240a887/html5/thumbnails/5.jpg)
Digital Literacy for Preschoolers 6/26/2015
Conference at McGill University 5
*Aim: Replicate RCT of teacher-delivered intervention to test effectiveness
*One whole (remote) school board (external validity)
*107 kindergarten and 96 grade 1 children in 24 classrooms
*10-12 hrs of teaching (close training and support), school board monitoring
*Treatment integrity and testing by board
Classroom-level randomization means need to run analysis at this level too:
HLM of results (pre-test as covariate) revealed significant main effects of letter-sound knowledge (p < .01) favoring ABRA
This analysis is highly conservative
*Effect size analyses (value-added d)
*Letter Sounds = +.66*Phonological Blending = +.52*Word Reading = +.52
*Effectiveness trial also a efficiency validity trial –the study accepted by teachers and officials
Four recent systematic reviews were chosen
• Slavin, Cheung, Groff & Lake (2008)
• Slavin, Lake, Chambers, Cheung, & Davis (2009)
• Torgenson & Zhu (2003)
• Andrews, Freeman, Hou, McGinn, Robinson & Zhu (2007)
• Used comparable review criteria• Use of randomized or matched control groups
• Study duration of at least 12 weeks
• Valid achievement measures
• Effect sizes, means or mean gain scores
• Found small to modest effect sizes
• Very little evidence in ICT effectiveness
“… the effects of supplementary computer-assisted instruction were small.”
Slavin, Cheung, Groff & Lake (2008)
“…instructional process programs designed to change daily teaching practices have substantially greater research support than programs that focus on curriculum or technology alone. ”
Slavin, Lake, Chambers, Cheung, & Davis (2009)
“These data would suggest that there is little evidence to support the widespread use of ICT in literacy learning in English.”
Torgenson & Zhu (2003)
“… we are thus unable to make confident comparisons between the effectiveness of different ICTs on learning in English for 5- to 16-year-olds.”
Andrews, Freeman, Hou, McGinn, Robinson, & Zhu (2007)
Archer et al. (2014) review reanalyzed 28 of the ICT effectiveness studies from the 4 systematic reviews deemed as high quality studies (all studies available).
2 variables were examined:
1. Reported quality of training and support that teachers received on implementing the ICT innovations
2. Reported quality and fidelity of implementation of the ICT innovations by teachers
![Page 6: Digital Literacy Preschoolers 6/26/2015 · Digital Literacy for Preschoolers 6/26/2015 Conference at McGill University 4 *Prototype (BLTK) built in 1998 *ABRACADABRA in 2002 (usability](https://reader035.vdocument.in/reader035/viewer/2022070903/5f5e89e652f74428e240a887/html5/thumbnails/6.jpg)
Digital Literacy for Preschoolers 6/26/2015
Conference at McGill University 6
• overall impression on process (were the procedures followed up to ensure correct implementation) and measuring tools
• 8 Questions
• Description of Implementation Training
• Details of the Intervention Implementation
Coding Scheme
0 = not present
1 = mentioned but NO information
2 = mentioned with limited detail
3 = mentioned with enough detail to roughly replicate
• 11 Questions
• Implementation Fidelity Process
• Implementation Fidelity Measurement Tool
• Results
• overall impression on the training section
Coding Scheme
0 = not present
1 = mentioned but NO information
2 = mentioned with limited detail
3 = mentioned with enough detail to roughly replicate
0
5
10
15
20
25
0 1 2 3
Training Support
0
5
10
15
20
25
30
0 1 2 3
Implementation Fidelity
0 = 53.8% 0 = 69.2%
N = 28
Kappa Implementation Fidelity = .65
p < .001
N = 28
Kappa Training Support = .88
p < .001
![Page 7: Digital Literacy Preschoolers 6/26/2015 · Digital Literacy for Preschoolers 6/26/2015 Conference at McGill University 4 *Prototype (BLTK) built in 1998 *ABRACADABRA in 2002 (usability](https://reader035.vdocument.in/reader035/viewer/2022070903/5f5e89e652f74428e240a887/html5/thumbnails/7.jpg)
Digital Literacy for Preschoolers 6/26/2015
Conference at McGill University 7
• Calculations used a main dataset
• Dataset reflected how the review papers structured the analysis e.g. if distinct ES for several studies in meta-analysis then several scores used here (n = 38 effect sizes)
• 16-teacher –led and 21 research student-led interventions, 1 parent-led intervention
*
00.20.40.60.8
11.2
Scor
e 0
Scor
e 1
Scor
e 2
Scor
e 3
Train/ Support
Train/Support
00.10.20.30.40.50.6
Fidelity
Fidelity
IV: training support DV: effect size
F(3,34) = 3.77, p=.019, =.249p2
Contrast Analysis
Code 1 vs. 0 p=.74
Code 2 vs. 0 p=.007
Code 3 vs. 0 p=.30
Training Support
IV: IF DV: effect size
F(2,35) = .89, p=.42, =.048
Contrast Analysis
Code 1 vs. 0 p=.21
Code 2 vs. 0 p=.80
Implementation Fidelity
p2
Reported Training and Support has big influence on Technology Effect Sizes for Literacy outcomes (explaining 26% of variance)
With n = 5 highly-ranked, ES ≈ 1 for rest ≈ 0.
Reported Treatment Integrity does not do so.
Why no effect for ‘Score 3’ studies?
All ‘score 3’ studies are reported in Campuzano et al. analysis of 10 technology products – training and support provided by ‘vendors’ of commercial products (as reported by teachers)
A single generic training and support paragraph is provided for all 10 programs, with wide variation (2 – 18 hours of training, over 50% with no additional support or training but not tied to specific ES scores)
The ‘implementation science’ of technology interventions adds a fresh perspective to (otherwise) pessimistic findings from existing meta-analyses:
There are grounds for optimism in well-trained and supported trials but data base of well-supported interventions here is small
Some recent studies show similar effects (Chambers et al., 2008; Ecalle et al., 2009; Savage et al., 2012; Wolgemuth et al., 2012;
We have not considered the absolute quality of technology here:
Grant et al. (2012) found few current technologies that conformed to best-evidence
Results need to be confirmed in formal meta-analysis.