description: tags: techappendix01 211

8/14/2019 description: tags: techappendix01 211

1/13


2/13


3/13

8WWC Intervention Report DaisyQuest September 28, 2006

Appendix A1.3 Study characteristics: Foster, Erickson, Foster, Brinkman, & Torgesen, 1994, Experiment 2: Kindergarten Classrooms(randomized controlled trial)

Characteristic Description

Study citation Foster, K. C., Erickson, G. C., Foster, D. F., Brinkman, D., & Torgesen, J. K. (1994). Computer administered instruction in phonological awareness: Evaluation of theDaisyQuest program.Journal of Research and Development in Education, 27 (2), 126137. (Experiment 2: Kindergarten Classrooms).

Participants Participants were 70 second-semester kindergarten students aged ve to seven years (average six years old) from four classrooms. Originally, nearly 97 students (all thestudents) from four kindergarten classrooms in a suburban elementary school were tested with the PPVT-R. Children with the highest and lowest scores were removed toreduce heterogeneity of the sample with regard to verbal ability. The 70 remaining children were matched in pairs according to their scores on the PPVT-R, with one of eachpair being randomly assigned to either the experimental group or the control group. One child from the experimental group changed schools and did not complete the study,for an analysis sample of 69.

Setting The study took place in a suburban elementary school.

Intervention Intervention students received 16 daily, 20-minuteDaisyQuest verbal training sessions in groups of four. Sessions took place at computers located in the hallway outside thechilds classroom under the guidance of an experimenter, who assisted students with their headphones and any computer glitches. A few of the children were absent fromseveral sessions, hence training time varied from 4.0 to 5.3 hours, with an average of 4.9 hours of training. This version ofDaisyQuest contained seven instructional activities.Students varied in the number of activities completed and speed with which they nished the activities.

Comparison The control group remained in their regular classroom, receiving their routine kindergarten instruction.

Primary outcomesand measurement

A series of tests were given at both pre- and posttesting: Screening Test of Phonological Awareness (STOPA),Undersea Challenge , Production Test of Blending, and Produc-tion Test of Segmenting. (See Appendix A2 for a more detailed description of outcome measures.)

Teacher training No information was given about teacher training, because teachers did not deliver the intervention.


4/13


5/13


6/13


7/13


Appendix A3 Summary of ndings for the alphabetics domain 1

Authors ndings from the studies

WWC calculationsMean outcome

(standard deviation 2)

Outcome measure Construct Study sampleSample size(students)

DaisyQuest group

Comparisongroup

Mean difference 3(DaisyQuest comparison)

Effectsize 4

Level ofstatistical

signicance(at = 0.05) 5

Improvementindex 6

Barker and Torgesen, 1995 (randomized controlled trial)

DaisyQuest compared with Hint and Hunt software (comparison 1)

Undersea Challenge Phonologicalawareness At-risk rst graders 49 10.49(1.10) 9.41(1.10) 1.08 0.96 Statisticallysignicant +33

Production Test of Segmenting Phonologicalawareness

At-risk rst graders 49 7.51(3.70)

3.27(2.90)

4.24 1.24 Statisticallysignicant

+39

Phoneme Elision Task Phonologicalawareness


2.41(1.70)

1.97 0.74 ns +27

Sound categorization Phonologicalawareness


8.43(4.60)

0.71 0.14 ns +6

Production Test of Blending Phonologicalawareness At-risk rst graders 49 7.30(4.20) 6.37(3.50) 0.93 0.23 ns +9

Phonological awareness average for comparison 1 in Barker & Torgesen, 1995 7 0.66 ns +25

Woodcock-Johnson WordIdentication subtest

Phonics At-risk rst graders 49 16.35(9.60)

11.59(6.40)

4.76 0.57 ns +22

Analog Reading Task Phonics At-risk rst graders 49 12.23(2.40)

12.12(2.40)

0.11 0.04 ns +2

Woodcock-Johnson Word

Analysis subtest

Phonics At-risk rst graders 49 2.92

(3.30)

1.28

(1.40)

1.64 0.63 ns +24

Experimental Non-Word Reading Phonics At-risk rst graders 49 21.84(9.80)

18.73(10.60)

3.11 0.30 ns +12

Phonics average for comparison 1 in Barker & Torgesen, 1995 7 0.39 ns +15

DaisyQuest compared with math software (comparison 2)

Undersea Challenge Phonologicalawareness


9.40(0.76)


+37

(continued)


8/13



WWC calculations

Mean outcome



DaisyQuest group

Comparisongroup


Effectsize 4

Level ofstatistical


Improvementindex 6

Phonics average for comparison 1 in Barker & Torgesen, 1995 7 (continued)



3.50(3.90)


+35

Phoneme Elision Task Phonologicalawareness


2.43(3.00)

1.95 0.61 ns +23

Sound categorization Phonologicalawareness


6.10(4.60)

3.04 0.60 ns +23

Production Test of Blending Phonologicalawareness


5.94(4.50)

1.36 0.31 ns +12

Phonological awareness average for comparison 2 in Barker & Torgesen, 1995 7 0.73 Statisticallysignicant

+27

Woodcock-Johnson WordIdentication subtest


12.39(8.40)

3.96 0.43 ns +17

Analog Reading Task Phonics At-risk rst graders 49 12.23(2.40)

12.56(2.30)

0.33 0.14 ns 5

Woodcock-Johnson Word Analysis subtest


2.03(2.90)

0.89 0.28 ns +11

Experimental Non-Word Reading Phonics At-risk rst graders 49 21.84(9.80)

19.38(9.80)

2.46 0.25 ns +10

Phonics average for comparison 2 in Barker & Torgesen, 1995 7 0.21 ns +8

Foster et al., 1994, Experiment 1: Child-care Facility (randomized controlled trial)

Phonological Awareness Test(PAT) (b)

Phonologicalawareness

5 year olds 27 22.40(3.10)

19.20(3.50)


+32

Screening Test of Phonological AwarenessExperimentalVersion (STOPA-E)


5 year olds 27 18.50(7.20)

12.40(6.50)


+31

(continued)

Appendix A3 Summary of ndings for the alphabetics domain 1 (continued)


9/13


Appendix A3 Summary of ndings for the alphabetics domain 1 (continued)


WWC calculations

Mean outcome



DaisyQuest group

Comparisongroup


Effectsize 4

Level ofstatistical


Improvementindex 6

Phonological awareness average for Foster et al., 1994, Experiment 1: Child-care Facility 7 0.90 Statisticallysignicant

+32

Foster et al., 1994, Experiment 2: Kindergarten Classrooms (randomized controlled trial)

Undersea Challenge Phonologicalawareness

Second-semesterkindergartners

69 53.30(9.00)

46.20(7.40)


+30

Screening Test of Phonological Awareness ( STOPA)



69 26.20(4.50)

25.30(7.40)

0.90 0.14 ns +6



69 11.80(2.60)

6.00(4.00)


+45

Production Test of Blending Phonologicalawareness


69 13.40(1.90)

10.80(3.80)


+30

Phonological awareness average for Foster et al., 1994, Experiment 2: Kindergarten Classrooms 7 0.89 Statisticallysignicant

+31

Mitchell & Fox, 2001 (randomized controlled trial)

DaisyQuest compared with teacher-delivered phonological awareness instruction (comparison 1)

Phonological Awareness Test(PAT) (a) total


Kindergartners andrst graders

69 73.20(10.31)

78.30(11.52)

5.10 0.46 ns 18

DaisyQuest vs. other software programs group (comparison 2)

Phonological Awareness Test(PAT) (a) total


Kindergartners andrst graders

69 73.20(10.31)

61.60(16.03)


+30

Domain averages for alphabetics

All studies 0.62 na +23

Individual studies

Barker & Torgesen, 1995 8 0.52 ns +20

Comparison 1 in Barker & Torgesen, 1995 8 0.54 ns +21

Comparison 2 in Barker & Torgesen, 1995 8 0.50 ns +19

(continued)


10/13
http://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdf


11/13
http://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdf


12/13


Appendix A4.2 Summary of subgroup ndings for the alphabetics domain 1


WWC calculationsMean outcome



DaisyQuest group

Comparisongroup


Effectsize 4

Level ofstatistical


Improvementindex 6

Foster et al., 1994, Experiment 2: Kindergarten Classrooms (randomized controlled trial)

Screening Test of Phonological Awareness ( STOPA)



14 lower achiev-ing students

(as dened bypretest STOPA)

22.90(7.30)

17.30(7.90)

5.60 0.69 ns +25

ns = not statistically signicant

1. This appendix presents subgroup ndings for measures that fall in the alphabetics domain. Total group scores were used for rating purposes and are presented in Appendix A3.2. The standard deviation across all students in each group shows how dispersed the participants outcomes are: a smaller standard deviation on a given measure would indicate that participants had more similar outcomes.3. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group.4. For an explanation of the effect size calculation, please see the Technical Details of WWC-Conducted Computations.5. Statistical signicance is the probability that the difference between groups is a result of chance rather than a real difference between the groups. The level of statistical signicance was calculated by the WWC and, where necessary,

corrects for clustering within classrooms or schools, for multiple outcomes within one domain, and for multiple comparison groups. For an explanation, see the WWC Tutorial on Mismatch. See the Technical Details of WWC-ConductedComputations for the formulas the WWC used to calculate statistical signicance. In the case of DaisyQuest, corrections for multiple outcomes and for multiple comparison groups were needed.

6. The improvement index represents the difference between the percentile rank of the average student in the intervention condition and that of the average student in the comparison condition. The improvement index can take on valuesbetween 50 and +50, with positive numbers denoting favorable results.
http://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdf


13/13


Appendix A5 DaisyQuest rating for the alphabetics domain

The WWC rates interventions as positive, potentially positive, mixed, no discernible effects, potentially negative, or negative. 1

For the outcome domain of alphabetics, the WWC rated DaisyQuest as having positive effects. The remaining ratings (potentially positive effects, mixed effects, no

discernible effects, potentially negative effects, and negative effects) were not considered because DaisyQuest was assigned the highest applicable rating.

Rating received

Positive effects: Strong evidence of a positive effect with no overriding contrary evidence.

Criterion 1: Two or more studies showing statistically signicant positive effects, at least one of which met WWC evidence standards for a strong design.Met. DaisyQuest had three studies showing statistically signicant positive effects, and all of these met WWC evidence standards for a strongdesign.

Criterion 2: No studies showing statistically signicant or substantively important negative effects.Met. The WWC analysis found no statistically signicant or substantively important negative effects in this domain.

1. For rating purposes, the WWC considers the statistical signicance of individual outcomes and the domain level effect. The WWC also considers the size of the domain level effect for ratings of

potentially positive effects. See the WWC Intervention Rating Scheme for a complete description.
http://ies.ed.gov/ncee/wwc/pdf/rating_scheme.pdfhttp://ies.ed.gov/ncee/wwc/pdf/rating_scheme.pdfhttp://ies.ed.gov/ncee/wwc/pdf/rating_scheme.pdfhttp://ies.ed.gov/ncee/wwc/pdf/rating_scheme.pdf

description: tags: techappendix01 211

Documents