![Page 1: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/1.jpg)
Evaluation of South Carolina’s Reading First
(SCRF) Initiative
Achievement and Survey Results from the First Three Years
Presented at the American Educational Research Association’s Annual Meeting
March 27, 2008
![Page 2: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/2.jpg)
Tammiee S. Dickenson, PhD
Robert L. Johnson, PhD
Heather L. Bennett, MSW
Katie A. Sesso-Dahlke, MSW
Brett Ermer
Mugdha Galande
Joanna Gilmore
Jessalyn Smith
Diane M. Monrad, PhD Sarah J. Gareau, MEdJennifer GayDiana MindrilaAnita Rawls
Pam WillsPatricia BranhamBecca DriggersDiane Dunham, EdS
![Page 3: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/3.jpg)
Overview
SCRF evaluation
Data sources
Achievement data
Participant group survey results
Conclusions
![Page 4: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/4.jpg)
SCDE
SCEPCOPE
Planning
Data Collection
Data Analysis
Reporting
SCRF Evaluation
![Page 5: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/5.jpg)
Participants
2004-2005: served 52 schools in 24 districts (approximately 12,000 students)
2005-2006: served 51 schools in 23 districts (approximately 11,000 students)
2006-2007: served 49 schools in 23 districts (approximately 10,500 students)
![Page 6: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/6.jpg)
Data Sources
Stanford Reading First assessment
Palmetto Achievement Challenge Test (PACT)
Participant group surveys
School Leadership Team (SLT) surveys
Professional development workshop surveys
![Page 7: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/7.jpg)
Annual SLT Presentation
Results presented from previous year
Stanford Reading First assessment results
Cohort analysis of all students in previous year
Matched analysis of students in all years
Individual school reports provided
Participant group survey results
Formative feedback for program improvement
![Page 8: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/8.jpg)
Stanford Reading First Assessment
Measures achievement in grades 1 through 3
Sections:
Multiple choice (components – phonemic awareness; phonics; vocabulary development; reading fluency; and reading comprehension strategies)
Oral fluency (components – speaking vocabulary and oral reading fluency)
![Page 9: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/9.jpg)
Results: At Grade Level (AGL) Grade 1
Fall Spring Annual Gain
Year 1 17.0% 48.2% 31.2%
Year 2 18.8% 51.2% 32.4%
Year 3 21.1% 54.3% 33.2%
Percentage of SCRF Grade 1 Students Scoring At Grade Level (AGL) on Stanford Reading First and Annual Gains
![Page 10: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/10.jpg)
Results: At Grade Level (AGL) Grade 2
Fall Spring Annual Gain
Year 1 30.8% 35.5% 4.7%
Year 2 35.7% 40.4% 4.7%
Year 3 38.0% 44.6% 6.6%
Percentage of SCRF Grade 2 Students Scoring At Grade Level (AGL) on Stanford Reading First and Annual Gains
![Page 11: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/11.jpg)
Results: At Grade Level (AGL) Grade 3
Fall Spring Annual Gain
Year 1 21.0% 41.5% 20.5%
Year 2 25.2% 46.2% 21.0%
Year 3 27.6% 49.0% 21.4%
Percentage of SCRF Grade 3 Students Scoring At Grade Level (AGL) on Stanford Reading First and Annual Gains
![Page 12: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/12.jpg)
Average NCE Scores on Stanford Reading First for SCRF Students with AGL Reference Line
0
10
20
30
40
50
60
Fall Gr1 Spr Gr1 Fall Gr2 Spr Gr2 Fall Gr3 Spr Gr3
SESAT2 Primary 1 Primary 2 Primary 3
NC
E S
co
re
At Grade Level SCRF Students
![Page 13: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/13.jpg)
Average Scale Scores on Stanford Reading First for SCRF Students and the Norm Group
400
450
500
550
600
650
700
Fall Gr1 Spr Gr1 Fall Gr2 Spr Gr2 Fall Gr3 Spr Gr3
SESAT2 Primary 1 Primary 2 Primary 3
Scal
e sco
re
Norm Group SCRF Students
![Page 14: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/14.jpg)
Participant Group Surveys
Respondent groups: interventionists, literacy coaches, principals, and teachers
Sections: implementation, level of support, roles and responsibilities, professional development, and program effectiveness
Comparison across years (2005-2006 compared to 2006-2007)
![Page 15: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/15.jpg)
Results: Participant Group Surveys Better understanding of program goals Reported a high rate of involvement in professional
development activities and found those activities helpful
Decrease in the need for professional development about the program and use of assessments
Increased need for professional development on effective instructional strategies to use for students below grade level
![Page 16: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/16.jpg)
Implementation Survey Items
0% 20% 40% 60% 80% 100%
I understand my rolesand responsibilities
within the SCRF Initiative.
I understand the goals ofthe SCRF Initiative.
I support the SCRFInitiative.
% Agree or Strongly Agree
Principal Literacy Coach Interventionist Teacher
![Page 17: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/17.jpg)
Implementation Survey Items (Cont.)
0% 20% 40% 60% 80% 100%
Teachers, interventionists,administrators, and the
school-based SCRF literacycoach(es) are working
together to implement theSCRF Initiative.
SLT members regularlyshare important information
with our faculty about theSCRF Initiative.
% Agree or Strongly Agree
Principal Literacy Coach Interventionist Teacher
![Page 18: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/18.jpg)
Describe One Benefit of the SCRF Initiative
Commonly cited benefits across groupsFocus on assessment
Increased awareness of students’ needs/strengths/weaknesses
Progress monitoring; Dominie Improved use of assessments; use of assessment data to
guide instruction
Resources Additional books and materials Extra funding/money
Professional development; study groups
Increased collaboration
Learned new strategies
![Page 19: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/19.jpg)
Results: Participant Recommendations
Recommendations
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
Changes to study groups Changes related toassessments
Increase writing emphasis
Per
cent
age
Teacher Interventionist Literacy Coach Principal
![Page 20: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/20.jpg)
Results: Effectiveness Ratings
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Not Effective Somewhat Effective Effective Very Effective
Effectiveness Rating
Per
cent
age
Teacher Interventionist Literacy coach Principal
83.1% of teachers rated SCRF as effective or very effective. 88.1% of interventionists rated SCRF as effective or very effective. 81.8% of literacy coaches rated SCRF as effective or very effective. 90.9% of principals rated SCRF as effective or very effective.
![Page 21: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/21.jpg)
Conclusions: Achievement
Students performed better on the Stanford Reading First assessment in the 2006-2007 school year as compared with the 2005-2006 school year.
Students’ scores on the Stanford Reading First assessment improved from fall to spring in all three years for all grade levels.
The largest gains were made by students in grade 1 and the lowest gains occurred in grade 2 in 2006-2007, which is also consistent with the last two years.
![Page 22: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/22.jpg)
Conclusions: Participant Groups Program participants have a better
understanding of program goals.
Participant groups find professional development useful.
Over 80% of each participant group rated the initiative as either effective or very effective in Years 2 and 3.
![Page 23: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/23.jpg)
Recommendations
More time for sustained reading for students in grade 2
Additional strategies for working with students below grade level
Participant groups’ suggestions regarding study groups
![Page 24: Evaluation of South Carolina’s Reading First (SCRF) Initiative Achievement and Survey Results from the First Three Years Presented at the American Educational](https://reader034.vdocument.in/reader034/viewer/2022051401/56649d3a5503460f94a15319/html5/thumbnails/24.jpg)
Contact InformationSouth Carolina Educational Policy CenterUniversity of South Carolina, College of EducationWardlaw Suite 010, Columbia, SC 29208
P: 803-777-8244
F: 803-777-0220 E: [email protected] (Diane Monrad)
Office of Program EvaluationUniversity of South Carolina, College of EducationWardlaw Suite 023, Columbia, SC 29208
P: 803-777-3402F: 803-777-8838E: [email protected] (Tammiee Dickenson)