program evaluation group2[1]
TRANSCRIPT
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 1/23
Warren T. Jackson Elementary1st Grade Accelerated Reader Program
Program EvaluationGroup 2: Katie Ingram,
TaMisha Kimble, and Stephanie Stone
ITEC 8435Fall 2009
Georgia Southern University
1
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 2/23
Table of Contents
Executive Summary………………………………………………………………… 3
Introduction …………………………………………………………………………. 5
Focus of the Evaluation ……………………………………………………………. 7
Brief Overview of Evaluation Plan and Procedures …………………………….. 10
Presentation of Evaluation Results ……………………………………………….. 11
Conclusions and Recommendations …………………………………………...… 15
Appendices …………………………………………………………………………...18
2
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 3/23
Executive Summary
This evaluation is to show that the Accelerated Reader Program is effective for all
students that participate in the program. Accelerated Reader is the most successful
and most popular reading program around. The students evaluated for this report are
first grade students. This report will inform you on how the program works. There is
detailed information to inform you on when and where students will have the opportunity
to take the quizzes. There are computers set up in each class with the adequate
software for students to take quizzes. Students will have the opportunity to take quizzes
throughout the day and they may access the software from their homes. This report will
show through STAR reading pre and post tests how AR improves students desire to
read, in turn, increasing their reading comprehension level.
There were several data sources used to help guide this evaluation. Parents were
surveyed before the evaluation and at the conclusion of this evaluation to see if they
have noticed in changes in their child’s reading habits and whether there is any
improvement. Data was also collected from AR goal history report. This report gives
goals for each student. The points earned by the student, their percentages accrued
and goal percentages are calculated on the report. Data from the STAR reading
assessment are used to help with data collection. Each student should show
improvement as far as reading ability and comprehension. This information can be
monitored by individual testing and teacher monitoring. Teachers should have
documentation of student’s improvements due to the AR program.
3
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 4/23
The evaluation findings prove that the program is working. There is data to show that
the students participating in the program are making great progress toward their goal.
By the end of the school year each student should make major progress in fluency and
comprehension due to the AR program. Students should develop lifelong reading
habits that will help them succeed in every subject area.
4
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 5/23
Introduction
This report is focusing on first grade students during the 2009-2010 school year. The
report will show that the AR program is working to help increase their fluency and
comprehension levels. These students should show an increase in reading abilities,
vocabulary knowledge, comprehension levels, and that reading helps learners in all
domains. These students are encouraged to read more. Teachers will find different
ways to monitor whether or not the students are doing what is required. In effort to
achieve No Child Left Behind schools across America feel it is necessary to use AR to
help boost student achievement. The administration, the board of education, teachers,
parents, and students are combining to increase students reading levels. Everyone
involved understands that reading makes students lifelong learners. This report is
aimed to show that the program is effective and it works. The students are achieving
their goals and their grades are increasing. Each first grade teacher has at least six
working computers in their class to make the quizzes accessible for all students. The
data collected is added to the report to help confirm accuracy.
Purpose of the Evaluation
The purpose of this evaluation is to show that the AR program is effective for all
students that are using the program. This report will show that the program increases
students desire to read, increases students comprehension of reading materials, helps
with fluency, motivate students to read whether at school or not, and produce lifelong
readers.
5
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 6/23
Audience for the Evaluation Report
This report is designed to for the use of the stakeholders which may include the
members of the board of education, principals of the school, the teachers, and the
media specialist. The report will also benefit the parents, students, and even the
program designers.
Limitations to the Evaluation
The limitations on this evaluation are very limited. The fact that the evaluation was
done quickly and could have been extended for a few more days would be considered a
major limitation. The time frame was short, but we feel the evaluation is adequate.
There are also some students that did not fully participate in the program. The fact that
some students and some parents did not fully do their parts could deter this evaluation.
Overview of Report Contents
The information in this report is as accurate as possible. This information is up-to-date
and is on file in this school system. The data is accessible to administrators and
teachers. The parents and students have a copy or can obtain a copy of the data for
their knowledge. The content of this report has been revised and data has been
checked multiple times to make sure the information is reported accurately.
The stakeholders are pleased with the program. The teachers and parents can see
positive results from each child that reads at least one book a week. The students are
6
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 7/23
motivated to set goals to help motivate them on how many points they will receive and
move up from there. The higher level the book, the more points students receive.
The AR program used by these students is Renaissance Learning which personalizes
reading practice based on student reading levels. This program is a little pricier than
some of the other programs, but it pays for itself. The quizzes are generated with
questions based on the books content and results can be seen through individual
student diagnostic reports.
7
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 8/23
Focus of the Evaluation
The Accelerated Reader is a progress-monitoring system that promotes guided
independent reading practice. Accelerated Reader provides quick and accurate
assessments of whether students have read and understood trade books they have
selected and textbook passages that have been assigned to them. It assigns point
values for each AR Reading Practice Quiz taken. Point values are a factor of book level
and performance on quizzes. Points accumulate during the school year and are a way
to monitor students’ reading volume. The system tracks student performance on AR
quizzes over time.
This evaluation will investigate the question: How is student participation in the
Accelerated Reader Program related to growth in the comprehension in first grade
students? The Accelerated Reader program in this first grade class is administered
throughout the day. Students have time during the day to take tests on AR books they
read at home and school. Each classroom has six student computers which are used
for taking Accelerated Reader tests. Students may only take tests at school but parents
may log in at any time to monitor their progress. Conducting this evaluation will help
teachers, media specialists, parents, and administrators decide if this program is
effective and how to steer it in a more productive direction if necessary.
In the beginning of the year students take the Star Test. The STAR Reading computer-
adaptive reading tests and database is an achievement level progress-monitoring
assessment that provides teachers with accurate reading scores for students in grades
8
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 9/23
K-12 in a matter of ten minutes. The individualized results of the STAR Test are taken
into account when students are assigned book levels and goals for point earnings.
The STAR test is administered in August, December, and May.
9
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 10/23
Brief Overview of Evaluation
In order to evaluate the effect of First Grade Accelerated Reader participation and
increased reading comprehension, information was obtained from student AR reports
and STAR tests.
In order to determine if students are effectively using the Accelerated Reader program,
quantitative data was collected from the class AR diagnostic reports. The diagnostic
report summarizes student performance on Reading Practice quizzes, shows progress
towards goals and identifies possible problems. Students should earn at least 1.5
points with an 85% or greater average each week. The pre and post STAR test scores
reflect the positive effect that AR has on student reading comprehension.
The STAR pre test data was reviewed to document the students reading levels
(Appendix A). The data descriptors allows teachers, administrators, and media
specialists to determine what levels they should be reading on, grade equivalence,
reading ability compared to others of the same grade level, instructional reading level,
and reading fluency (Appendix B). The Accelerated Reader goal report is based on the
STAR pre test (Appendix C). The STAR post tests show that all students reading levels
have increased since pre test (Appendix D). In all data, 21 first graders reading levels
were assessed.
10
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 11/23
Presentation of Evaluation Results
Students were given a pre test using the STAR system prior to beginning the
Accelerated Reader (AR) program.
STAR Pre-Test Data
Student Test DateGP SS GE PR PR LevelNCE IRL ORF
1 8/18/09 1.00 306 2.6 94 91-96 82.7 2.6 832 8/18/09 1.00 94 1.3 65 39-72 58.1 PP 273 8/18/09 1.00 321 2.7 95 92-97 84.6 2.7 864 8/18/09 1.00 160 1.7 79 73-85 67.0 1.0 435 8/18/09 1.00 64 0.6 20 10-34 32.3 PP 96 8/18/09 1.00 67 0.7 29 15-41 38.3 PP 127 8/18/09 1.00 276 2.4 92 88-95 79.6 2.3 75
8 8/18/09 1.00 42 0.0 1 1-3 1.0 PP 09 8/18/09 1.00 239 2.2 89 85-92 75.8 1.9 6510 8/18/09 1.00 131 1.5 74 67-80 63.5 P 3711 8/18/09 1.00 77 0.9 49 39-56 49.5 PP 1912 8/18/09 1.00 237 2.2 89 85-92 75.8 1.8 6413 8/18/09 1.00 59 0.4 10 3-24 23.0 PP 514 8/18/09 1.00 60 0.4 10 3-26 23.0 PP 615 8/18/09 1.00 69 0.8 34 20-45 41.3 PP 1316 8/18/09 1.00 68 0.7 0 0-0 0.0 PP 1217 8/18/09 1.00 252 2.3 90 85-94 77.0 2.0 6818 8/18/09 1.00 64 0.6 20 10-34 32.3 PP 919 8/18/09 1.00 62 0.5 15 6-29 28.2 PP 8
20 8/18/09 1.00 137 1.7 81 75-86 68.5 1.1 4721 8/18/09 1.00 87 1.2 60 43-68 55.3 PP 25
Students are then given an individual goal for the AR program based upon their scores
from the STAR test. This goal is appropriate for each individual student for them to be
successful at their independent reading level. The AR program then keeps track of their
goal and points earned toward their goal. If students meet at least 50% of their goal,
they are considered to be effectively participating in the AR program.
11
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 12/23
Goal Report
Student PointsGoal
PointsEarned
% of Goal
1 72.0 27.2 37.82 54.0 52.3 96.93 73.0 81.4 111.54 58.0 22.1 38.15 50.0 14.2 28.46 50.0 19.6 39.27 68.0 33.3 49.08 50.0 9.9 19.89 66.0 44.4 67.310 56.0 62.8 112.111 50.0 23.4 46.812 66.0 32.3 48.913 50.0 20.3 40.614 50.0 12.3 24.615 50.0 15.9 27.616 50.0 9.8 19.617 67.0 27.1 40.418 50.0 27.6 55.219 50.0 33.7 67.420 58.0 33.4 57.521 53.0 29.4 55.5
* Eight out of 21 students (38%) are considered to be effectively using the AR program.
12
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 13/23
After a period of approximately four months, students are given another test, post test,
using the STAR system. Scores from the pre test and post test are then compared to
see if there has been any growth in the students’ reading levels and whether or not their
goals will be increased for the AR program.
STAR Post-Test Data
Student Test DateGP SS GE PR PR LevelNCE IRL ORF
1 12/1/09 1.30 434 3.7 98 97-99 93.3 3.6 1422 12/1/09 1.30 353 3.0 95 91-97 84.6 3.0 923 12/1/09 1.30 392 3.4 97 95-98 89.6 3.3 1084 12/1/09 1.30 227 2.1 83 77-87 70.1 1.7 625 12/1/09 1.30 92 1.3 52 14-61 51.1 PP 9
6 12/1/09 1.30 105 1.4 58 25-65 54.2 PP 127 12/1/09 1.30 321 2.7 93 90-95 81.1 2.7 868 12/1/09 1.30 79 0.9 33 20-43 40.7 PP 09 12/1/09 1.30 339 2.8 94 91-97 82.7 2.9 8910 12/1/09 1.30 253 2.3 86 81-91 72.8 2.0 6911 12/1/09 1.30 254 2.3 87 81-91 73.7 2.0 6912 12/1/09 1.30 290 2.5 91 86-94 78.2 2.4 8013 12/1/09 1.30 112 1.5 60 35-67 55.3 PP 3214 12/1/09 1.30 114 1.5 61 38-67 55.9 PP 3315 12/1/09 1.30 82 1.0 38 25-49 43.6 PP 2216 12/1/09 1.30 70 0.8 14 7-25 27.2 PP 1417 12/1/09 1.30 296 2.6 91 86-94 78.2 2.5 81
18 12/1/09 1.30 124 1.5 64 51-71 57.5 PP 3519 12/1/09 1.30 103 1.4 59 24-65 54.8 PP 3020 12/1/09 1.30 225 2.1 83 76-87 70.1 1.7 6121 12/1/09 1.30 87 1.2 60 43-68 55.3 PP 25
Summary
13
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 14/23
GP SS GE PR NCE IRL ORF
PretestMean
0.95 138 1.6 55 52.8 P 34
PosttestMean
1.30 209 2.0 77 65.3 1.5 58
Change +0.35 +71 +0.4 +22 +12.5 +1.2 +24
*An examination of the summary report concludes a positive trend in all reading areasis shown between the pre-test and post-test for all 21 students.
Stakeholder Perceptions
Stakeholders in this evaluation have shown an overwhelming positive response to the
AR program. When interviewed, stakeholders in all areas, from administration to
parents, had positive comments about the AR program. They have seen the positive
trends in all reading areas to include vocabulary, comprehension, reading levels,
stamina, and fluency, and conclude that the AR program is working effectively when all
students participate to the fullest extent of the program. The only negative comments
came from teachers who feel that due to the limited number of computers in the
classroom as well as time restraints during the day can cause some students to not use
the program to its maximum potential.
14
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 15/23
Conclusion and Recommendations
Based upon the evaluation data of the effectiveness of the AR program on all students,
this evaluation study concludes:
Program Strengths:
· Vocabulary increases
· Reading levels increase
· Fluency levels in reading increase
· Reading comprehension is increased
· Stamina is increased
Program Weaknesses:
· Time restraints during the day
· Lack of computers available
· Some students do not fully participate in the program
Recommendations:
· Set a specific testing time in daily schedule
· Sign up for a designated time to take tests in the computer lab or media
center (if available)
· Utilize parent volunteers to help with “traffic flow” on and off computers for
tests
15
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 16/23
· Allow students extra time when finished with work to take a test
· Utilize buddy system to help motivate students who are not fully
participating
· Create a reward system to motivate students who are not fully
participating
Therefore, it is the recommendation of the evaluation study that the positive impact and
effectiveness of the AR program on all students shows cause to continue the use of the
program in schools that currently use it. If possible, the effectiveness would be
increased if students who were not fully participating be able to have a buddy to help
remind them to read and take a test. It may also be beneficial to incorporate some sort
of reward system in order to motivate students to fully participate. It is also
recommended that a set time in daily schedules may help to more students to
participate. In conjunction, a set time in a computer lab or media center, if possible,
where there are more computers to access may also help in participation levels.
Finally, it is recommended that possible use of parent volunteers to help with the flow of
“traffic” on and off the computers may allow for more timely use and afford more
students the opportunity to take full advantage of the AR program.
16
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 17/23
References
Renaissance Learning. 2009.Renaissance learning. Retrieved December 2, 2009 from
https://hosted115.renlearn.com/51385/ .
17
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 18/23
Appendix A
STAR Pre-Test Data
Student Test DateGP SS GE PR PR LevelNCE IRL ORF
1 8/18/09 1.00 306 2.6 94 91-96 82.7 2.6 832 8/18/09 1.00 94 1.3 65 39-72 58.1 PP 273 8/18/09 1.00 321 2.7 95 92-97 84.6 2.7 864 8/18/09 1.00 160 1.7 79 73-85 67.0 1.0 435 8/18/09 1.00 64 0.6 20 10-34 32.3 PP 96 8/18/09 1.00 67 0.7 29 15-41 38.3 PP 127 8/18/09 1.00 276 2.4 92 88-95 79.6 2.3 758 8/18/09 1.00 42 0.0 1 1-3 1.0 PP 09 8/18/09 1.00 239 2.2 89 85-92 75.8 1.9 6510 8/18/09 1.00 131 1.5 74 67-80 63.5 P 37
11 8/18/09 1.00 77 0.9 49 39-56 49.5 PP 1912 8/18/09 1.00 237 2.2 89 85-92 75.8 1.8 6413 8/18/09 1.00 59 0.4 10 3-24 23.0 PP 514 8/18/09 1.00 60 0.4 10 3-26 23.0 PP 615 8/18/09 1.00 69 0.8 34 20-45 41.3 PP 1316 8/18/09 1.00 68 0.7 0 0-0 0.0 PP 1217 8/18/09 1.00 252 2.3 90 85-94 77.0 2.0 6818 8/18/09 1.00 64 0.6 20 10-34 32.3 PP 919 8/18/09 1.00 62 0.5 15 6-29 28.2 PP 820 8/18/09 1.00 137 1.7 81 75-86 68.5 1.1 4721 8/18/09 1.00 87 1.2 60 43-68 55.3 PP 25
18
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 19/23
Appendix B
Definitions
March 2009 Reproducible Form © 2009 Renaissance Learning, Inc. Page 1 of 2
STAR Reading scores represent how students performed on the test compared with theperformance of a nationally representative sample of students, called the norms group.These scores present a snapshot of achievement at a specific point in time. As with anytest, it is important to remember that many factors can affect a student’s test scores.STAR Reading test scores give only one picture of how a student is doing in school.
Scaled score (SS) is useful for comparing student performance over time and acrossgrades. A scaled score is calculated based on the difficulty of questions and the number
of correct responses. Because the same range is used for all students, scaled scorescan be used to compare student performance across grade levels. STAR Readingscaled scores range from 0 to 1400. All norm-referenced scores are derived from thescaled score.
Percentile rank (PR) is a norm-referenced score that provides a measure of a student’sreading ability compared to other students in the same grade nationally. The percentilerank score, which ranges from 1 to 99, indicates the percentage of other studentsnationally who obtained scores equal to or lower than the score of a particular student.For example, a student with a percentile rank score of 85 performed as well as or better than 85 percent of other students in the same grade.
Percentile rank range (PR Range) indicates the statistical variability in a student’spercentile rank score. For example, a student with a percentile rank range of 32–59 islikely to score within that range if the STAR Reading test is taken again within a shorttime (i.e., four to six weeks).
Normal curve equivalent (NCE) is a norm-referenced score that is similar to percentilerank, but is based on an equal interval scale. This means the difference between anytwo successive scores on the NCE scale has the same meaning throughout the scale.NCEs are useful in making comparisons between different achievement tests and for statistical computations—for example, determining an average score for a group of
students. NCE scores range from 1 to 99 and are mostly used for research.
Grade equivalent (GE) is a norm-referenced score that represents how a student’s testperformance compares with other students nationally. For example, a fifth-gradestudent with a GE score of 7.6 performed as well as a typical seventh-grader after thesixth month of the school year. This score doesn’t necessarily mean that the student iscapable of reading seventh-grade material—it only indicates that the student’s readingskills are well above average for the fifth grade.
19
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 20/23
Grade placement (GP) is a numeric representation of a student’s grade level, based onthe specific month in which a student takes a STAR Reading test. STAR Readingconsiders the standard school year to run from September through June and assignsincrement values of 0.0 through 0.9 to these months. The software automaticallyassigns grade placements using a student’s grade level and the month in which a STAR
Reading test was taken. GP is important because PR and NCE values are based notonly on the Scaled Score but also on the grade placement of the student at the time of the test.
Instructional reading level (IRL) is a criterion-referenced score that indicates thehighest reading level at which a student is at least 80 percent proficient at recognizingwords and understanding material with instructional assistance. For example, aseventh-grade student with a score of 8.0 reads eighth-grade words with 80 percentaccuracy or better. IRL scores are Pre-Primer (PP), Primer (P), grades 1.0 through12.9, and Post-High School (PHS).
Estimated oral reading fluency (Est. ORF) is an estimate of a student’s ability to readwords quickly and accurately in order to comprehend text efficiently. Students with oralreading fluency demonstrate accurate decoding, automatic word recognition, andappropriate use of the rhythmic aspects of language (e.g., intonation, phrasing, pitch,and emphasis). Est. ORF is reported in correct words per minute, and is based on aknown relationship between STAR Reading performance and oral reading fluency.
20
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 21/23
Appendix C
Goal Report
Student Points
Goal
Points
Earned
% of
Goal1 72.0 27.2 37.82 54.0 52.3 96.93 73.0 81.4 111.54 58.0 22.1 38.15 50.0 14.2 28.46 50.0 19.6 39.27 68.0 33.3 49.08 50.0 9.9 19.89 66.0 44.4 67.310 56.0 62.8 112.111 50.0 23.4 46.8
12 66.0 32.3 48.913 50.0 20.3 40.614 50.0 12.3 24.615 50.0 15.9 27.616 50.0 9.8 19.617 67.0 27.1 40.418 50.0 27.6 55.219 50.0 33.7 67.420 58.0 33.4 57.521 53.0 29.4 55.5
21
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 22/23
Appendix D
STAR Post-Test Data
Student Test DateGP SS GE PR PR LevelNCE IRL ORF
1 12/1/09 1.30 434 3.7 98 97-99 93.3 3.6 1422 12/1/09 1.30 353 3.0 95 91-97 84.6 3.0 923 12/1/09 1.30 392 3.4 97 95-98 89.6 3.3 1084 12/1/09 1.30 227 2.1 83 77-87 70.1 1.7 625 12/1/09 1.30 92 1.3 52 14-61 51.1 PP 96 12/1/09 1.30 105 1.4 58 25-65 54.2 PP 127 12/1/09 1.30 321 2.7 93 90-95 81.1 2.7 868 12/1/09 1.30 79 0.9 33 20-43 40.7 PP 09 12/1/09 1.30 339 2.8 94 91-97 82.7 2.9 8910 12/1/09 1.30 253 2.3 86 81-91 72.8 2.0 6911 12/1/09 1.30 254 2.3 87 81-91 73.7 2.0 6912 12/1/09 1.30 290 2.5 91 86-94 78.2 2.4 80
13 12/1/09 1.30 112 1.5 60 35-67 55.3 PP 3214 12/1/09 1.30 114 1.5 61 38-67 55.9 PP 3315 12/1/09 1.30 82 1.0 38 25-49 43.6 PP 2216 12/1/09 1.30 70 0.8 14 7-25 27.2 PP 1417 12/1/09 1.30 296 2.6 91 86-94 78.2 2.5 8118 12/1/09 1.30 124 1.5 64 51-71 57.5 PP 3519 12/1/09 1.30 103 1.4 59 24-65 54.8 PP 3020 12/1/09 1.30 225 2.1 83 76-87 70.1 1.7 6121 12/1/09 1.30 87 1.2 60 43-68 55.3 PP 25
22
8/8/2019 Program Evaluation Group2[1]
http://slidepdf.com/reader/full/program-evaluation-group21 23/23
Appendix E
STAR Pre/Post Test Summary Report
GP SS GE PR NCE IRL ORF
PretestMean
0.95 138 1.6 55 52.8 P 34
PosttestMean
1.30 209 2.0 77 65.3 1.5 58
Change +0.35 +71 +0.4 +22 +12.5 +1.2 +24
*Numbers based on 21 students in class.