team read improving literacy in the seattle school district
DESCRIPTION
Team Read Improving Literacy in the Seattle School District. PA 590C – Program Evaluation Fall 2007 Anne Debuyserie. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/1.jpg)
Team Read Team Read Improving Literacy in the Seattle Improving Literacy in the Seattle
School DistrictSchool District
PA 590C – Program EvaluationPA 590C – Program EvaluationFall 2007Fall 2007
Anne DebuyserieAnne Debuyserie
![Page 2: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/2.jpg)
“...a Seattle School district tutoring program dedicated to increasing the reading skills of elementary students through year-long coaching by high school students ... The program's goals for participating high-school tutors are to develop work experience, a sense of responsibility and accomplishment, and to learn the rewards of community service.”
(Team Read Mission Statement, 1998)
![Page 3: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/3.jpg)
What is Team Read?What is Team Read?
Private-public partnership to improve Private-public partnership to improve literacy in Seattle Public Schoolsliteracy in Seattle Public Schools
Cross-age tutoring program using Cross-age tutoring program using high school student as tutorshigh school student as tutors
Started in March 1998 with a three-Started in March 1998 with a three-phase implementationphase implementation
June 1999: 10 schools, 335 students, June 1999: 10 schools, 335 students, 300 coaches300 coaches
June 2000: addition of 7 schoolsJune 2000: addition of 7 schools
![Page 4: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/4.jpg)
Team Read- Org ChartTeam Read- Org Chart
![Page 5: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/5.jpg)
Team Read Stakeholders Team Read Stakeholders
Craig McCaw (and wife Susan): BusinessmanCraig McCaw (and wife Susan): Businessman John Stanford: Seattle Schools SuperintendentJohn Stanford: Seattle Schools Superintendent Joan Dore: Seattle Schools Reading SpecialistJoan Dore: Seattle Schools Reading Specialist Tricia McKay: Team Read Program MgrTricia McKay: Team Read Program Mgr Team Read Advising Board: Seattle School Team Read Advising Board: Seattle School
District Staff, Alliance for Education pgm District Staff, Alliance for Education pgm representatives, Mc Caws, community representatives, Mc Caws, community representatives.representatives.
Additional investors joined by year 2.Additional investors joined by year 2.
![Page 6: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/6.jpg)
Who is Team Read?Who is Team Read? CoachesCoaches
– Strict selection guidelinesStrict selection guidelines– High expectations from Team Read High expectations from Team Read
leadersleaders– CompensationCompensation
ReadersReaders– School selection comes firstSchool selection comes first– Student selection based on reading levelStudent selection based on reading level– Eligibility based on needEligibility based on need– First-come, first-serve basisFirst-come, first-serve basis– Low socio-economic backgroundLow socio-economic background
![Page 7: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/7.jpg)
Who is Team Read?Who is Team Read? Site CoordinatorsSite Coordinators
– Part-time, teachers or school staffPart-time, teachers or school staff– Meet informally with Tricia McKay for Meet informally with Tricia McKay for
feedbackfeedback VolunteersVolunteers
– Needed due to amount of workNeeded due to amount of work– Partnership with VISTA/Americorps Partnership with VISTA/Americorps
and U. of Washington Pipelineand U. of Washington Pipeline– Assist site coordinatorsAssist site coordinators
![Page 8: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/8.jpg)
Team Read SessionTeam Read Session Two 1 hr session per week during school Two 1 hr session per week during school
yearyear Standardized session format and structureStandardized session format and structure
– 3.00 PM: coaches arrive3.00 PM: coaches arrive– 3.15 PM: snack with reader3.15 PM: snack with reader– 3.30 pm: reading session begins3.30 pm: reading session begins– 4.15 PM: reading session ends/ coaches fill out 4.15 PM: reading session ends/ coaches fill out
paperworkpaperwork– 4.30 PM: reading coaches leave4.30 PM: reading coaches leave
Opportunity for site coordinators to be Opportunity for site coordinators to be creativecreative
![Page 9: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/9.jpg)
Team Read Evaluation ResultsTeam Read Evaluation Results 11stst year evaluation: not as positive as year evaluation: not as positive as
expectedexpected 22ndnd year: promising but Team Read’s year: promising but Team Read’s
impact not as promising as hopedimpact not as promising as hoped Tricia McKay disappointed by results Tricia McKay disappointed by results
and wonders if findings provide her and wonders if findings provide her with the information needed for with the information needed for improvementsimprovements
Questions about the accuracy of Questions about the accuracy of results.results.
![Page 10: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/10.jpg)
Team Read EvaluationsTeam Read Evaluations Evaluation conducted by Margo Jones, Evaluation conducted by Margo Jones,
statistician by trade.statistician by trade. 17 schools in the program by June 2000 17 schools in the program by June 2000
but only 10 used in the evaluation.but only 10 used in the evaluation. 3 main objectives to show Team Read’s 3 main objectives to show Team Read’s
Impact:Impact:– Goal#1: Do the reading skills of the student
readers improve significantly during their participation in the Team Read program?
– Goal#2: How does the program affect the reading coaches?
– Goal#3 What is working well, and what can be improved?
![Page 11: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/11.jpg)
Evaluation Methods Goal #1Evaluation Methods Goal #1 Goal #1: Reading skills. Two-fold Goal #1: Reading skills. Two-fold
approachapproachGradeGrade Pre-TestPre-Test Post-TestPost-Test Post-Test MetricPost-Test Metric
22 DRA Fall 1999DRA Fall 1999 DRA Spring DRA Spring 20002000 Test LevelTest Level
33 DRA Spring DRA Spring 1999*1999*
ITBS Spring ITBS Spring 2000*2000* NCENCE
44 ITBS Spring ITBS Spring 19991999
WASL Spring WASL Spring 20002000 Scale ScoreScale Score
55 WASL Spring WASL Spring 19991999
ITBS Spring ITBS Spring 20002000 NCENCE
![Page 12: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/12.jpg)
Apples and orangesApples and orangesGradGrad
ee Pre-TestPre-Test Post-TestPost-Test Post-Test Post-Test MetricMetric
22
33 NCENCE
44
55
![Page 13: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/13.jpg)
Evaluation Methods Goal #1, Evaluation Methods Goal #1, Cont’dCont’d
22ndnd method: determine proportion of method: determine proportion of Team Read participants who moved Team Read participants who moved from below grade level test score to from below grade level test score to at-or above level at-or above level compared to all compared to all students from entire districtstudents from entire district
Note: one of the selection criteria to Note: one of the selection criteria to be in Team Read is to be within be in Team Read is to be within bottom bottom 25% of district reading 25% of district reading test scoretest score
![Page 14: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/14.jpg)
Evaluation Methods Goal #1Evaluation Methods Goal #1 Both methods for goal 1 are flawed.Both methods for goal 1 are flawed. Note from Jones in report confirming problems Note from Jones in report confirming problems
with different tests.with different tests. “The question then becomes ‘Is it reasonable to
assume that the pre- and post-tests measure the same skills?’ The evaluator adopted a correlation criterion for answering this question. If the correlation between pre- and post-tests was near or above .8 (see footnote 5) (as was the case for the 2nd grade, where pre- and post-tests were essentially the same), the pre- to post-test change score was interpreted as a gain score. If the correlation coefficient was much less than .8, no analysis was performed.”
![Page 15: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/15.jpg)
Evaluation Methods, Goals #1- Evaluation Methods, Goals #1- AssessmentAssessment
Visible problems with tests:Visible problems with tests:– Pre and post-tests do not measure the Pre and post-tests do not measure the
same skills except in 2same skills except in 2ndnd grade grade– All Team Read participants are within All Team Read participants are within
the bottom quartile of the reading the bottom quartile of the reading test scores and can’t be compared test scores and can’t be compared with students district-wide.with students district-wide.
– Analysis could not be carried out for Analysis could not be carried out for some grades (3some grades (3rdrd) due to ) due to incompatibility of reading testsincompatibility of reading tests
![Page 16: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/16.jpg)
Evaluation Method, Goal #2Evaluation Method, Goal #2
Method: questionnaire given to coaches Method: questionnaire given to coaches during their last week of coaching. during their last week of coaching.
Goal: to assess the impact of Team Goal: to assess the impact of Team Read on coaches.Read on coaches.
Identify 3 main areas of job satisfaction:Identify 3 main areas of job satisfaction:– How positive is their experience with TR?How positive is their experience with TR?– Extent to which the coaches feel that TR Extent to which the coaches feel that TR
helps students readershelps students readers– How supportive is TR’s site coordinators How supportive is TR’s site coordinators
and staff?and staff?
![Page 17: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/17.jpg)
Evaluation Method, Goal #2 - Evaluation Method, Goal #2 - AssessmentAssessment
Jones results showed that coaches were Jones results showed that coaches were satisfied with their job and felt a sense satisfied with their job and felt a sense of pride and accomplishment.of pride and accomplishment.
Test performed by Jones was successful Test performed by Jones was successful in meeting its goals, unlike in Goal #1.in meeting its goals, unlike in Goal #1.
Some questions provided answers as to Some questions provided answers as to what to improve in the program.what to improve in the program.
Test is useful to the program managerTest is useful to the program manager
![Page 18: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/18.jpg)
Evaluation Method, Goal #3Evaluation Method, Goal #3
Method: interviews with major Method: interviews with major stakeholders, one site visit, review stakeholders, one site visit, review of last 5 years’ research literature of last 5 years’ research literature on cross-age tutoring and last 3 on cross-age tutoring and last 3 years research on best practices.years research on best practices.
Goal: assess the overall progress Goal: assess the overall progress of the program.of the program.
![Page 19: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/19.jpg)
After the EvaluationAfter the Evaluation Margo Jones makes her Margo Jones makes her
recommendations based on her recommendations based on her findings to improve the efficacy of findings to improve the efficacy of Team Read.Team Read.
Tricia McKay reads the evaluation Tricia McKay reads the evaluation but fail to see what the results are but fail to see what the results are telling her about the programtelling her about the program
![Page 20: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/20.jpg)
Evaluation Methods - SummaryEvaluation Methods - SummaryGoalsGoals Methodology
Reading improvementamong elementarystudents
Analyzed standardized test scores for 10 of 17 Team Read schools. Rate of increase on pre- and post- tests. Percent of students moving from below reading level to at/above grade level. Collected data by grade level and by school. Compared Team Read results to district scores.
Coach development Surveyed coaches by questionnaireProgram implementation Interviewed stakeholders
Observed Team Read sessionReviewed other studies
![Page 21: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/21.jpg)
Evaluation ProblemsEvaluation Problems Academic v. Practitioner approach.Academic v. Practitioner approach. Result presentation.Result presentation. Research design flaws.Research design flaws.
– Not all the data was analyzedNot all the data was analyzed– Comparison with dissimilar itemsComparison with dissimilar items– Statistical analysis stretchedStatistical analysis stretched
Evaluation only covers 3 goals.Evaluation only covers 3 goals. Lack of communication between the Lack of communication between the
evaluator and the program evaluator and the program manager.manager.
![Page 22: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/22.jpg)
What Could Have Been Done What Could Have Been Done Better?Better?
![Page 23: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/23.jpg)
Impact: 2005-2006 Reading Results
From Team Read’s 2006 Annual Report.http://www.teamread.com/downloads/team_read_106903_report_proof.pdf
![Page 24: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/24.jpg)
Team Read Update - 2006Team Read Update - 2006 Program Evaluation Results for 2005-2006
School Year – 70% of 2nd graders and 52% of 3rd graders were
reading at/above or approaching grade level – 54% of 2nd graders and 35% of 3rd graders
gained greater than 1.5 grade levels in reading – 98% of the parents of 2nd & 3rd graders and 92%
of their teachers reported increased reading skills as a result of participation in Team Read
– 76% of 2nd & 3rd graders said that reading was more fun since joining Team Read
![Page 25: Team Read Improving Literacy in the Seattle School District](https://reader033.vdocument.in/reader033/viewer/2022050820/56815d54550346895dcb5f0e/html5/thumbnails/25.jpg)