description: tags: techappendix01 210

8/14/2019 description: tags: techappendix01 210

1/7


2/7


3/7

10WWC Intervention Report Accelerated Reader/Reading Renaissance April 23, 2007

Appendix A3.1 Summary of study findings included in the rating for the comprehension domain1

Authors findings from the study2

WWC calculationsMean outcome

(standard deviation3)

Outcome measureStudy

sample

Sample size(teachers/students)

AR/RR

groupComparison

group

Mean difference4

(AR/RRcomparison) Effect size5

Statisticalsignificance6

(at = 0.05)Improvement

index7

Ross, Nunnery, & Goldfeder, 2004 (randomized controlled trial) 8

STAR Reading Test Grade 3 13/178 389.5

(139.6)

336.8

(198.3)

52.70 0.31 ns +12

Average9 for comprehension (Ross, Nunnery, & Goldfeder, 2004) 0.31 ns +12

ns = not statistically significant

1. This appendix reports findings considered for the effectiveness rating and the improvement index.

2. The authors adjusted posttest scores for pretest dif ferences between study groups.

3. The standard deviation across all students in each group shows how dispersed the participants outcomes are: a smaller standard deviation on a given measure would indicate that part icipants had more similar outcomes.

4. Positive differences and effect sizes favor the intervention group; negative differences and effect sizes favor the comparison group.

5. For an explanation of the effect size calculation, see Technical Details of WWC-Conducted Computations.

6. Statistical significance is the probability that the difference between groups is a result of chance rather than a real difference between the groups.

7. The improvement index represents the difference between the percentile rank of the average student in the intervention condition versus the percentile rank of the average student in the comparison condition. The improvement index

can take on values between 50 and +50, with positive numbers denoting results favorable to the intervention group.8. The level of statistical significance was reported by the study authors or, where necessary, calculated by the W WC to correct for clustering within classrooms or schools and for multiple comparisons. For an explanation about the

clustering correction, see the WWC Tutorial on Mismatch. See Technical Details of WWC-Conducted Computations for the formulas the WWC used to calculate statistical significance. In the case of Ross, Nunnery, & Goldfeder (2004), a

correction for clustering was needed, so the significance levels may differ from those reported in the original study.

9. The WWC-computed average effect sizes are simple averages rounded to two decimal places and the average improvement indices are calculated from the average effect size. In this case, the average is the same as the single out-

come measure for the single study.
http://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdf


4/7
http://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdfhttp://ies.ed.gov/ncee/wwc/pdf/mismatch.pdfhttp://ies.ed.gov/ncee/wwc/pdf/conducted_computations.pdf


5/7


Appendix A4.1 Accelerated Reader/Reading Renaissancerating for the comprehension domain

The WWC rates an interventions eects in a given outcome domain as positive, potentially positive, mixed, no discernible eects, potentially negative, or negative.1

For the outcome domain o comprehension, the WWC ratedAccelerated Reader/Reading Renaissance as having potentially positive eects. It did not meet the

criteria or having positive eects because there was only one study or this domain. The remaining ratings (mixed eects, no discernable eects, potentially negative

eects, negative eects) were not considered because Accelerated Reader/Reading Renaissance was assigned a higher applicable rating.

Rating received

Potentially positive effects: Evidence o a positive eect with no overriding contrary evidence.

Criterion 1: At least one study showing a statistically signifcant or substantively importantpositive eect.

Met.Accelerated Reader/Reading Renaissance had one study showing a substantively important positive eect on the comprehension domain.

Criterion 2: No studies showing a statistically signifcant or substantively important negative eect and ewer or the same number o studies showing indeterminate

eects than showing statistically signifcant or substantively important positive eects.

Met.Accelerated Reader/Reading Renaissance had no studies showing negative or indeterminate eects on the comprehension domain.

Other ratings considered

Positive effects: Strong evidence o a positive eect with no overriding contrary evidence.

Criterion 1: Two or more studies showing statistically signifcantpositive eects, at least one o which met WWC evidence standards or a strong design.

Not met.Accelerated Reader/Reading Renaissance had only one study that met WWC evidence standards.

Criterion 2: No studies showing statistically signifcant or substantively importantnegative eects.

Met. No studies showed statistically signifcant or substantively important negative eects.

1. For rating purposes, the WWC considers the statistical signifcance o individual outcomes and the domain-level eect. The WWC also considers the size o the domain-level eect or ratings o

potentially positive or potentially negative eects. See the WWC Intervention Rating Scheme or a complete description.
http://ies.ed.gov/ncee/wwc/pdf/rating_scheme.pdfhttp://ies.ed.gov/ncee/wwc/pdf/rating_scheme.pdf


6/7


Appendix A4.2 Accelerated Reader/Reading Renaissancerating for the general reading achievement domain

The WWC rates an interventions eects in a given outcome domain as positive, potentially positive, mixed, no discernible eects, potentially negative, or negative.1

For the outcome domain o general reading achievement, the WWC rated Accelerated Reader/Reading Renaissance as having potentially positive eects. It did not

meet the criteria or having positive eects because there was only one study or this domain. The remaining ratings (mixed eects, no discernable eects, potentially

negative eects, negative eects) were not considered becauseAccelerated Reader/Reading Renaissance was assigned a higher applicable rating.

Rating received

Potentially positive effects: Evidence o a positive eect with no overriding contrary evidence.

Criterion 1: At least one study showing a statistically signifcant or substantively importantpositive eect.

Met.Accelerated Reader/Reading Renaissance had one study showing a statistically signifcant positive eect on the general reading achievement

domain.

Criterion 2: No studies showing a statistically signifcant or substantively important negative eect and ewer or the same number o studies showing indeterminateeects than showing statistically signifcant or substantively important positive eects.

Met.Accelerated Reader/Reading Renaissance had no studies showing negative or indeterminate eects on the general reading achievement

domain.

Other ratings considered

Positive effects: Strong evidence o a positive eect with no overriding contrary evidence.

Criterion 1: Two or more studies showing statistically signifcantpositive eects, at least one o which met WWC evidence standards or a strong design.

Not met.Accelerated Reader/Reading Renaissance had only one study that met WWC evidence standards.

Criterion 2: No studies showing statistically signifcant or substantively importantnegative eects.

Met. No studies showed statistically signifcant or substantively important negative eects.

1. For rating purposes, the WWC considers the statistical signifcance o individual outcomes and the domain-level eect. The WWC also considers the size o the domain-level eect or ratings o

potentially positive or potentially negative eects. See the WWC Intervention Rating Scheme or a complete description.
http://ies.ed.gov/ncee/wwc/pdf/rating_scheme.pdfhttp://ies.ed.gov/ncee/wwc/pdf/rating_scheme.pdf


7/7


Appendix A5 Extent of evidence by domain

Sample size

Outcome domain Number of studies Schools Students Extent of evidence1

Alphabetics 0 0 0 na

Fluency 0 0 0 na

Comprehension 1 not specified 178 Small

General reading achievement 1 not specified 394 Small

na = not applicable/not studied

1. A rating o moderate to large requires at least two studies and two schools across studies in one domain and a total sample size across studies o at least 350 students or 14 classrooms.

Otherwise, the rating is small.

description: tags: techappendix01 210

Documents