stu.westga.edustu.westga.edu/~ksharp1/8480_evalreport_kas.docx · web viewone of the reasons the...
TRANSCRIPT
Running Head: EVALUATION REPORT 1
Fountas & Pinnell Benchmark Assessment System
Prepared by Kimberly A. Sharp
Prepared for Yvonne Frey and Ivy Creek Elementary School
November 1, 2010
EVALUATION REPORT 2
Executive Summary
Background Information
Ivy Creek Elementary School is a seven year old school located in Buford, Georgia. It is
comprised of 886 students in grades PreK-Fifth Grade with the following demographics: 58%
Caucasian, 20% Hispanic, 10% African – American, 8% Asian and 4% Multi-Racial. The
majority of students come from families with an average socioeconomic status.
The school was recently invited to pilot the Fountas & Pinnell Benchmark Assessment
System largely as a result of the efforts of the reading coach at Ivy Creek Elementary School.
The Fountas & Pinnell Benchmark Assessment Program is designed to measure student’s
reading levels with the use of benchmark books. The information gained from a personalized
assessment session with each student provides the teacher with valuable insight into the reading
ability of the student and the concepts that need to be addressed during reading instruction
(Fountas & Pinnell, 2008.)
Purpose
The administrative staff at Ivy Creek Elementary School requested that this program be
evaluated to determine the impact of the assessment system on reading instruction and student
achievement in reading. This assessment system is closely aligned with the district’s views of
reading instruction. The Language Arts Department feels confident that the Fountas & Pinnell
Benchmark Assessment Program meets the needs of reading teachers.
Now that this formative evaluation project is complete, the leaders at Ivy Creek
Elementary School can examine how the Fountas & Pinnell Benchmark Assessment System is
impacting classroom teachers and their reading instruction. Additional professional learning
opportunities will be offered to teachers based on the needs outlined in this evaluation.
EVALUATION REPORT 3
Evaluation Questions
What impact do professional learning sessions have on teachers as they
implement the Fountas & Pinnell Benchmark Assessment System?
How does the Fountas & Pinnell Benchmark Assessment System impact reading
instruction?
How does the Fountas & Pinnell Benchmark Assessment System compare to the
assessment program used last year?
Should the Fountas & Pinnell Benchmark Assessment System be adopted for use
at Ivy Creek Elementary School next year?
Methods
All 37 certified staff members who administer the Fountas & Pinnell Benchmark
Assessment System at Ivy Creek Elementary were asked to participate in this study. Qualitative
data was gathered from open-ended and closed-ended questions in surveys (Appendix A) and
one member from each grade level was interviewed using the interview guide approach
(Appendix B.) Informal student assessment data was used to determine if students appear to be
progressing in the area of reading achievement. Because the formal assessment was not
administered before the end of this project, the evaluator examined student work and relied on
teacher observation to determine if students are showing signs of achievement.
Key Findings
The results of the survey indicated that teachers were pleased overall with the training
they received and that the initial professional development session during pre-planning was the
most helpful. Several teachers indicated that they especially appreciated watching the video of
an expert conducting an actual assessment and comparing the expert results with their own. All
EVALUATION REPORT 4
of the teachers interviewed expressed a desire to spend more time with the supplemental
materials in the kit.
According to the survey, teachers found that the Fountas & Pinnell Benchmark
Assessment Program was implemented smoothly. In each of the subsequent interviews, teachers
shared that the results of their assessments were extremely accurate. They were pleased with the
program in that it has enabled them to form solid reading groups and paint a truer picture of each
student’s reading ability. Notably, they described that the assessment period was originally
scheduled to last for two weeks, but it actually took four weeks to complete. Teachers also
explained that they believed this process would be faster next year because all of the students
who were tested this year would begin with a consistent starting point.
One of the reasons the assessments took so long was a result of the inaccuracies with the
Word Lists. This was a significant source of frustration for many teachers. However, the
teachers did like the recording forms as a communication tool to use during parent-teacher
conferences. They especially found that having copies of the recording forms and assessments
available in the teacher workrooms was helpful for logistical reasons.
The majority of teachers found that they liked the Fountas & Pinnell Benchmark
Assessment Program as much as or more than programs they have used in the past. They cited
the accuracy of the results and the amount of information gathered for each student as a benefit.
They also liked the fact that the assessment books were shorter with more complex text.
The survey showed that teachers like this program enough to agree with adopting the
program at Ivy Creek and within Gwinnett County. There were no noticeable differences in the
responses of teachers when cross-referenced by age, teaching experience and degree.
EVALUATION REPORT 5
Recommendations
Training and Professional Development
Professional development instructors should take into consideration the meeting schedule
as well as the number of days available to prepare for the beginning of school. Training should
be adjusted to account for experiential differences by allowing teachers to attend a beginning,
intermediate or advanced session based on their needs. When possible, instructors should use an
actual assessment as a model for teachers attending training.
Future professional learning sessions should be focused on the contents of the assessment
kits and the use of the supplemental materials provided by the Fountas & Pinnell Benchmark
Assessment Program. It would be appropriate to offer a variety of scheduled sessions (i.e.
before or after school, summer, etc.) in an effort to accommodate everyone.
Implementation and Reading Instruction
Additional professional learning on how to practically use the error information in
reading instruction would be valuable to teachers. In order to overcome the time constraints of
teachers trying to assess all students in a timely fashion, perhaps classes could be covered by
support staff or substitute teachers for a period of time so that teachers’ schedules were free to
conduct assessments during the first two weeks of school. Professional development could also
be offered to share practical strategies for conducting assessments throughout the school day.
The issue of mid-term grading is a concern that should also be addressed by grade levels.
A common strategy for reporting student progress needs to be developed. Informal assessments
revealed that students had improved at least one or two levels since the beginning of the school
year. However, it is unclear if this was because of the strategies implemented as a result of the
EVALUATION REPORT 6
Fountas & Pinnell Benchmark Assessment Program, the efforts of the teachers conducting
quality reading instruction or a combination of both.
Resource materials that accompany the program need to be adjusted as well. The
recording forms and assessments need to be modified so that they are easier to read. It was
suggested that these documents be placed on the Share Drive for all teachers to easily access in
addition to providing copies in the teacher workrooms.
Finally, Ivy Creek should invest in three more kits to share with intermediate grade
levels. This will eliminate the need for sharing kits amongst grade levels during the limited time
available for assessments to be completed.
Comparison to Other Programs
While teachers liked the variety of texts offered by this program, they would like to have
several options available so that students are looking at a new text for each of the upcoming
assessments. This may be beyond the control of the local school or county officials, but it is a
suggestion worth offering to the company producing the Fountas & Pinnell Benchmark
Assessment Program materials.
Adoption for Ivy Creek and Gwinnett
Overall, teachers recommended that the Fountas & Pinnell Benchmark Assessment
Program be adopted by Ivy Creek Elementary and by Gwinnett County. There was some
confusion about this portion of the survey that will need to be addressed with teachers during the
presentation of the findings of this evaluation report. I believe that the confusion was due to the
teachers’ familiarity with the work of Fountas & Pinnell. However, contrary to the opinion of
some, Ivy Creek had not utilized the Fountas & Pinnell Benchmark Assessment Program in the
past.
EVALUATION REPORT 7
Conclusion
The key findings of this evaluation report on the Fountas & Pinnell Benchmark
Assessment Program were overwhelmingly positive. Teachers at Ivy Creek Elementary School
indicated that they were pleased with the professional development sessions, the quality of the
materials, the impact the program had on reading instruction and student achievement.
Generally, they felt that this assessment program was as good as or better than programs they
had previously used and they recommended that the Fountas & Pinnell Benchmark Assessment
Program be adopted for continued use at Ivy Creek Elementary School and within Gwinnett
County. While the initial indications are very positive, by following the recommendations
mentioned above, the school has an opportunity to make the Fountas & Pinnell Benchmark
Assessment Program even more beneficial to teachers.
Based on the findings in this report, I believe that the Fountas & Pinnell Benchmark
Assessment Program has the potential for improving reading instruction and student achievement
at Ivy Creek Elementary. Professional development will be more focused on the specific needs
of teachers now that the general instruction is complete. As teachers grow more comfortable
with utilizing the program, I believe they will become more efficient in conducting their
assessments and they will begin to apply what they learn about student errors to their reading
instruction. Ultimately, both the students and teachers will benefit from the implementation of
this program.
EVALUATION REPORT 8
Evaluation Report: Fountas & Pinnell Benchmark Assessment System
Introduction
Background Information
Ivy Creek Elementary School is located in northeastern Gwinnett County in Buford,
Georgia. Founded in 2003, the school is comprised of 886 students in grades PreK-Fifth Grade.
At a recent faculty meeting, the principal of Ivy Creek, Yvonne Frey, shared statistics about the
students attending the school. Demographically, the student population is 58% Caucasian, 20%
Hispanic, 10% African – American, 8% Asian and 4% Multi-Racial. Approximately 29% of
students receive free or reduced price lunch while the majority of students come from families
with average socioeconomic status (Y. Frey, personal communication, May 17, 2010).
Gwinnett County prides itself on becoming a system of world-class schools. As a
member of the largest school district in the state of Georgia, Ivy Creek Elementary has enjoyed
the support of an active Language Arts Department. The school was recently invited to pilot the
Fountas & Pinnell Benchmark Assessment System largely as a result of the efforts of the reading
coach at Ivy Creek Elementary School.
The administrative staff at Ivy Creek Elementary School requested that this program be
evaluated to determine the impact of the assessment system on reading instruction and student
achievement in reading. This evaluation was designed to provide valuable information to the Ivy
Creek Elementary School Stakeholders: Principal, Assistant Principals, Reading Coach,
Teachers and Staff. In addition, the results of the evaluation are available for review by the
county office staff. These data may have an impact on whether the program is recommended to
all elementary schools in the county. As the media specialist at Ivy Creek Elementary School
EVALUATION REPORT 9
and one of the few teachers not directly involved with administering the Fountas & Pinnell
Benchmark Assessment System, I conducted an objective evaluation of the program.
Evaluation Program
Fountas and Pinnell have spent more than twenty years establishing a leveling system for
educators to use when evaluating students’ reading ability. Teachers are able to determine
whether a student is meeting the standards set for each grade level by using leveled books and
measuring a student’s reading fluency and comprehension. Levels follow a twenty-six point
scale from the easiest level (A) to the most challenging level (Z.) These alphabetical levels
correspond with reading ability levels ranging from Kindergarten to 8th Grade (Fountas &
Pinnell, 2008.)
The Fountas & Pinnell Benchmark Assessment Program is designed to measure student’s
reading levels with the use of benchmark books. These leveled books are used during a one-on-
one session between the teacher and the student. During these sessions, the student reads the
book aloud and discusses the book while the teacher observes and codes the behavior of the
reader. This system of coding allows the teacher to identify student errors with regards to
fluency and comprehension. The information gained from this personalized assessment session
provides the teacher with valuable insight into the reading ability of the student and the concepts
that need to be addressed during reading instruction (Fountas & Pinnell, 2008.)
Purpose
The purpose of conducting an evaluation of the Fountas & Pinnell Benchmark
Assessment System at Ivy Creek Elementary School was to determine the impact it has on
reading instruction and student achievement. The program is designed so that teachers
administering the assessment are able to use student errors to guide instruction. As a result,
EVALUATION REPORT 10
reading instruction is more focused and allows students to progress more rapidly through the
stages of reading.
This assessment system is closely aligned with the district’s views of reading instruction.
Specifically, the program correlates with the use of guided reading and reading workshops as
methods of delivery. In a program study of a reading initiative designed to improve reading
achievement, Al Otaiba and Hosp recommended that assessment programs should align with
district reading programs (2008.) The Language Arts Department feels confident that the
Fountas & Pinnell Benchmark Assessment Program meets the needs of reading teachers.
Unfortunately, complete student achievement data was unavailable during the time
constraints for this project. While there is some indication of student progression in reading,
official midterm and year-end assessments will not be administered until January and again in
May. Therefore, the focus of this project was on professional learning related to implementing
the assessment system and the impact this program is having on teacher instruction.
Now that this formative evaluation project is complete, the leaders at Ivy Creek
Elementary School have a true picture of how the Fountas & Pinnell Benchmark Assessment
System is impacting classroom teachers and their reading instruction. Additional professional
learning opportunities will now be offered to teachers based on the needs outlined in this
evaluation. This training will be provided to teachers prior to the administration of the midterm
and year-end assessments. The data collected during the course of this project has also provided
professional learning instructors with specific feedback on how to further guide teachers in
administering effective reading instruction.
These steps reinforce the ideas suggested by Engstrom and Danielson (2006) in their
review of a professional development program on Howard Gardner’s multiple intelligence
EVALUATION REPORT 11
theory. They found that teachers need ongoing support when implementing a new program.
Additionally, they recommended that action research be conducted throughout program
implementation so that professional development could be focused on the needs of teachers. It is
also critical for teachers to have sufficient time and resources to understand the intent of new
standards and the structure of new curriculum (Obara & Sloan, 2009).
Once the needs of teachers have been identified, professional development opportunities
can be designed to address those needs. It is important to narrow the focus of these sessions to
the most relevant topics while integrating specific content and teaching ideas. It is also
necessary to provide teachers with networking time and long-term support (Hutchins, Arbaugh,
Abell, Marra, & Lee, 2008) to ensure that the implementation of the Fountas & Pinnell
Benchmark Assessment Program is done correctly.
After teachers have administered the mid-term and post-evaluations, the evaluator and the
client plan to review student reading achievement data; CRCT scores, Guided Reading Levels,
and Reading Standards. These hard data should serve to reinforce the qualitative data obtained
as a result of this evaluation project (Kaufman, Guerra, & Platt, 2006.) Although these data will
be obtained beyond the scope of this evaluation, it will be necessary to include student
achievement results in order to complete the full program evaluation. This summative evaluation
is scheduled for completion by the end of the 2010-11 school year.
Evaluation Questions
In order to provide meaningful information to Ivy Creek Elementary School stakeholders
through this evaluation, the following questions were answered:
What impact do professional learning sessions have on teachers as they implement
the Fountas & Pinnell Benchmark Assessment System?
EVALUATION REPORT 12
How does the Fountas & Pinnell Benchmark Assessment System impact reading
instruction?
How does the Fountas & Pinnell Benchmark Assessment System compare to the
assessment program used last year?
Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at
Ivy Creek Elementary School next year?
MethodsParticipants
All certified staff members who administer the Fountas & Pinnell Benchmark
Assessment System were asked to participate in this study. This group included thirty-three
classroom teachers in Grades 1-5, the reading coach, one intervention specialist and two gifted
reading teachers.
Design and Procedures
A variety of activities were designed to address all of the objectives outlined in this
evaluation report. A qualitative research approach was used in that data was gathered from
teacher surveys (Appendix A) and interviews (Appendix B). Questions were designed to
evaluate teachers’ perceptions of their professional learning experience as related to the Fountas
& Pinnell Benchmark Assessment System. Additional questions addressed the impact the
program had on the way teachers delivered reading instruction. Comparing the Fountas &
Pinnell Benchmark Assessment System to the reading programs used in years past was an
important factor in determining whether or not to proceed with the adoption of the Fountas &
Pinnell model. Qualitative data was gathered from open-ended and closed-ended questions in
both surveys and in-person interviews.
EVALUATION REPORT 13
Instruments
A comprehensive survey was developed by the evaluator for approval by the client.
Because only one survey was administered for the purpose of this project, a select group of three
teachers was asked to review the survey for clarity. All certified staff members responsible for
reading instruction were asked to complete the survey. Questions were both open and closed-
ended. Twenty-eight of the thirty-three teachers responded to the survey for a response rate of
85%.
Once the surveys were complete, in-person interviews were conducted with five teachers
who were willing to participate. One member from each grade level was interviewed using the
interview guide approach (Appendix B.) This qualitative data collection method allowed for
topics and issues to be specified in advance while allowing for variance in the sequence and
wording of the questions as the interview progressed. This method kept the interview
conversational which perhaps elicited more meaningful data (Johnson & Christensen, 2007).
Finally, informal student assessment data was used to determine if students appeared to
be progressing in the area of reading achievement. Because the formal assessment was not
administered before the end of this project, the evaluator examined student work and relied on
teacher observation to determine if students were showing signs of achievement.
Objective Survey Interview Informal Student Assessment
What impact do professional learning sessions have on teachers as they implement the Fountas & Pinnell Benchmark Assessment System?
x x
How does the Fountas & Pinnell Benchmark
x x
EVALUATION REPORT 14
Assessment System impact reading instruction?How does the Fountas & Pinnell Benchmark Assessment System compare to the assessment program used last year?
x x x
Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at Ivy Creek Elementary School next year?
x x x
Summary of Key Findings
What impact do professional learning sessions have on teachers as they implement the Fountas & Pinnell Benchmark Assessment System?
Pre-Plan
ning
Grade L
evel P
lanning
One-On-O
ne Trai
ningDVDs
Adequate
Train
ing012345 4.32 3.89 3.93
3.144.39
Training and Professional Development
Survey Question
Liker
t Sca
le
The results of the survey indicated that teachers were pleased overall with the training
they received and that the initial professional development session during pre-planning was the
most helpful. Several teachers indicated that they especially appreciated watching the video of
an expert conducting an actual assessment and comparing the expert results with their own. The
training sessions also offered detailed instructions on completing running records. While this
was helpful to teachers new to the process, at least three teachers commented that this session
EVALUATION REPORT 15
should have been optional for those who were already proficient at completing running records.
Another classroom teacher suggested that it might have been helpful to hear a “short and sweet”
version of the training including a sample assessment and details about how this program was
different from the programs used in the past. Some of the information presented was redundant.
All of the teachers interviewed expressed a desire to spend more time with the
supplemental materials in the kit. They commented that many materials available to them have
been left unused. They would like time to explore these resources and to dig deeper into what
the Fountas & Pinnell Assessment System has to offer. At the time of the survey, many had not
watched the DVDs. However, at the time of the individual interviews, they had seen them and
found them to be helpful.
Comments in both the surveys and interviews indicated that additional training should be
centered around using the coding system within running records (MSV) and on how to proceed
with grouping students once you have determined student levels and how they progress into
higher groups.
Beginning of the Year
Summer Throughout the Year
0
1
2
3
4
54.11
2.323.07
Timing for Professional Development
Survey Question
Liker
t Sca
le
EVALUATION REPORT 16
By far, the majority of teachers prefer to receive this type of training at the beginning of
the school year. Several comments in the interviews revealed concern over trying to attend to
such important training during pre-planning. Because of the furlough days, there was less time
to prepare for the first day of school and that was a distraction for many trying to learn how to
implement the Fountas & Pinnell Benchmark Assessment Program. Some teachers indicated
they like to receive training throughout the school year. Additionally, some teachers expressed a
preference for half-day planning while others asked for training in smaller chunks of time
throughout the year. These questions received the widest range of responses, but overall the
survey showed that teachers disagreed with attending training during the summer.
How does the Fountas & Pinnell Benchmark Assessment System impact reading instruction?
User Fr
iendly
Accurat
e Rea
ding Leve
l
Guides Instr
uction
Understan
d Erro
rs
Use Stu
dent E
rrors
Word Lists
Record
ing Form
s
Copies in
Workr
oom012345 4.11 4.07 4.39 3.96 4.21
3.32 3.714.61
Implementation and Reading Instruction
Survey Question
Liker
t Sca
le
According to the survey, teachers found that the Fountas & Pinnell Benchmark
Assessment Program was implemented smoothly. In each of the subsequent interviews, teachers
shared that the results of their assessments were extremely accurate. They were pleased with the
program in that it has enabled them to form solid reading groups and paint a truer picture of each
student’s reading ability. Teachers found that the levels of text were appropriate with
EVALUATION REPORT 17
challenging vocabulary. Those interviewed indicated that all of their students had improved one
or two levels since the initial testing at the beginning of the school year.
The survey results also indicated that teachers are using the results of the Fountas &
Pinnell Benchmark Assessment Program to guide instruction and that it helps them understand
student errors and incorporate those errors into their guided reading instruction. However, in the
interviews, the teachers expressed a desire to use the assessment information in this way, but
they had not effectively implemented this concept. Notably, they described that the assessment
period was originally scheduled to last for two weeks, but it actually took four weeks to
complete. One teacher commented, “I am concerned with the amount of instructional time it
took to conduct 27 assessments. I started all of them too low initially, and will need to make
adjustments for next year.” Another expressed concern over grading. “Mid-term grades are due
and there are few grades due to the amount of time it takes to implement the reading workshop
program and to test the students’ independent reading levels.” Teachers explained that they
believed this process would be faster next year because all of the students who were tested this
year would begin with a consistent starting point.
One of the reasons the assessments took so long was a result of the inaccuracies with the
Word Lists. This was a significant source of frustration for many teachers. Comments included,
“The Word Lists were off by at least two levels for almost every student,” and “The Word Lists
do not adequately equate with the student comprehension level. Many times students can word
call from a list but when they read that particular level book, they struggle.” Because the Word
Lists were used as a tool for determining on what level to begin assessing a student, the process
took longer than intended to complete each individual assessment.
EVALUATION REPORT 18
The teachers liked the recording forms as a communication tool to use during parent-
teacher conferences. It took some getting used to the layout of the forms, but they found that the
fluency rubric was easy to code and was more objective than previous programs. They
especially found that having copies of the recording forms and assessments available in the
teacher workrooms was helpful for logistical reasons. The only complaint about these materials
was that the font size was too small.
Another concern shared by second, third and fourth grade teachers was that they really
needed access to both the upper and lower grade kits. Some students were reading significantly
above or below grade level and they found themselves asking other teachers to borrow materials
in order to properly assess their students. Those teachers needed their materials, so it was
difficult to coordinate sharing them so that all assessments could be completed in a timely
fashion.
How does the Fountas & Pinnell Benchmark Assessment System compare to the assessment program used last year?
More Infor-mation
More Accurate Prefer F&P0
1
2
3
4
53.89 3.86 3.93
Comparison with Other Programs
Survey Question
Liker
t Sca
le
The majority of teachers found that they liked the Fountas & Pinnell Benchmark
Assessment Program as much as or more than programs they have used in the past. They cited
EVALUATION REPORT 19
the accuracy of the results and the amount of information gathered for each student as a benefit.
They were also pleased with the fact that the books are shorter with more complex text. One
teacher indicated that this program was better because it did not just test the student’s stamina for
reading. Another shared that the variety of text was a nice change of pace, but she wished that
there was more than one book offered at each level so that students were exposed to different
texts during subsequent testing. Another teacher liked that the reading levels in this program
stretched well above grade level so that her students were assessed more accurately.
Several teachers expressed that the Fountas & Pinnell Benchmark Assessment Program
was more time-consuming to administer than other programs they had used. One shared that she
didn’t think it was any better or worse. “It's the same type of assessment with a different
company's name on it.”
Should the Fountas & Pinnell Benchmark Assessment System be adopted for use at Ivy Creek Elementary School next year?
Adopt at ICES Adopt in Gwinnett0
1
2
3
4
54.11 4.14
Recommendation for Adoption
Survey Question
Liker
t Sca
le
This question seemed to prompt some confusion amongst survey participants and yet they
obviously like this program enough to agree with adopting the program at Ivy Creek and within
EVALUATION REPORT 20
Gwinnett County. However, some believed that the program had been adopted in 2003 when
reading workshop was implemented during Ivy Creek’s first year. Others were concerned that
we had spent money on a program that might not be used in the future. “I would like for us to
find ONE program and use it consistently. It seems like we are continuously changing mid
stream which makes it difficult to use the information in a meaningful way. The constant change
also has an effect on teacher moral because we feel like we are always learning how to
implement a "new and better" method.”
Recommendations
Training and Professional Development
Teachers found that receiving initial training at the beginning of the school year during
pre-planning was most beneficial as the assessments needed to be completed as soon as the
school year began. However, those leading professional development should take into
consideration the meeting schedule for the week as well as the number of days available to
prepare for the beginning of school. In the event of a shortened pre-planning week, it is possible
that teachers could receive training during the first week of school.
Professional development instructors should also keep in mind that teachers will have
varying degrees of experience with conducting the assessments and running records. Training
should be adjusted to account for these differences by allowing teachers to attend a beginning,
intermediate or advanced session based on their needs. These sessions should include any
differences between the assessments that teachers are currently using and the Fountas & Pinnell
Benchmark Assessment Program. When possible, use an actual assessment as a model for
teachers attending training.
EVALUATION REPORT 21
Future professional learning sessions should be focused on the contents of the assessment
kits and the use of the supplemental materials provided by the Fountas & Pinnell Benchmark
Assessment Program. Teachers expressed an interest in find out more information about
effectively using the coding system within running records (MSV), grouping students and using
errors for instruction. Preferences as to the length and timing of these sessions varies widely. It
would be appropriate to offer different options in an effort to accommodate everyone.
Implementation and Reading Instruction
Survey results indicated that teachers are effectively implementing the Fountas & Pinnell
Benchmark Assessment Program. However, personal interviews with teachers from each grade
level revealed that while they have a desire to utilize student errors in their lesson planning and
reading instruction, the reality is that this is not happening. I believe that the discrepancy is a
result of teachers answering the questions with what they feel they are supposed to be doing as
opposed to what they are actually doing. To overcome this discrepancy, additional professional
learning on how to practically use the error information in reading instruction would be valuable
to teachers.
One of the biggest challenges with implementing the Fountas & Pinnell Benchmark
Assessment Program was the time it took for teachers to conduct the assessments. While
teachers believe that this process will go more quickly next year, it is important to address this
issue. Perhaps classes could be covered by support staff or substitute teachers for a period of
time so that teachers’ schedules were free to conduct assessments during the first two weeks of
school. Professional development could also be offered to share practical strategies for
conducting assessments throughout the school day.
EVALUATION REPORT 22
The issue of mid-term grading is a concern that should also be addressed by grade levels.
Because of the length of time needed to complete the assessments, some teachers found it
difficult to have enough grades to accurately report reading progress. A common strategy for
reporting student progress needs to be developed. During the interviews, teachers did indicate
that all of their students had improved at least one or two levels since the beginning of the school
year. However, it is unclear as to the specific reason for the improvement. It could have been
because of the strategies implemented as a result of the Fountas & Pinnell Benchmark
Assessment Program, but it is more likely the efforts of the teachers conducting quality reading
instruction or a combination of both. On teacher remarked that this assessment program was just
another “tool in the toolbox” to reach students and teach them how to read.
Resource materials that accompany the program need to be adjusted as well. While the
Word Lists were a critical component to finding a starting point for assessments this year,
teachers should be able to rely on end of year assessment levels as a starting point for next year.
This would make the obstacles encountered with the Word Lists obsolete next year. However,
the recording forms and assessments need to be adjusted so that they are easier to read. It was
suggested that these documents be placed on the Share Drive for all teachers to easily access in
addition to providing copies in the teacher workrooms.
Finally, Ivy Creek should invest in three more kits to share with intermediate grade
levels. Second grade teachers need a kit for Grades 3-5 while third and fourth grade teachers
need kits for Grades K-2. This will eliminate the need for sharing kits amongst grade levels
during the limited time available for assessments to be completed.
EVALUATION REPORT 23
Comparison to Other Programs
The majority of teachers found that they liked the Fountas & Pinnell Benchmark
Assessment Program as much as or more than programs they have used in the past. While they
liked hearing a variety of texts read by each student, they would like to have several options
available so that students are looking at a new text for each of the upcoming assessments. This
may be beyond the control of the local school or county officials, but it is a suggestion worth
offering to the company producing the Fountas & Pinnell Benchmark Assessment Program
materials.
Adoption for Ivy Creek and Gwinnett
Overall, teachers recommended that the Fountas & Pinnell Benchmark Assessment
Program be adopted by Ivy Creek Elementary and by Gwinnett County. There was some
confusion about this portion of the survey that will need to be addressed with teachers during the
presentation of the findings of this evaluation report. I believe that the confusion was due to the
teachers’ familiarity with the work of Fountas & Pinnell. The leveled book room at Ivy Creek
has been organized using the Fountas & Pinnell leveling system, but teachers used a variety of
assessment programs (i.e. DRA, Rigby) to determine student reading levels. A correlation chart
that matched student reading levels in other programs to the Fountas & Pinnell levels was
provided so that teachers could easily locate materials in the leveled book room. However, there
was previously no consistency in the leveling systems used to assess student reading levels
across grade levels and Ivy Creek had not utilized the Fountas & Pinnell Benchmark Assessment
Program in the past.
EVALUATION REPORT 24
Conclusion
The key findings of this evaluation report on the Fountas & Pinnell Benchmark
Assessment Program were overwhelmingly positive. Teachers at Ivy Creek Elementary School
indicated that they were pleased with the professional development sessions, the quality of the
materials, the impact the program had on reading instruction and student achievement.
Generally, they felt that this assessment program was as good as or better than programs they
had previously used and they recommended that the Fountas & Pinnell Benchmark Assessment
Program be adopted for continued use at Ivy Creek Elementary School and within Gwinnett
County. While the initial indications are very positive, by following the recommendations
mentioned above, the school has an opportunity to make the Fountas & Pinnell Benchmark
Assessment Program even more beneficial to teachers.
Based on the findings in this report, I believe that the Fountas & Pinnell Benchmark
Assessment Program has the potential for improving reading instruction and student achievement
at Ivy Creek Elementary. Professional development will be more focused on the specific needs
of teachers now that the general instruction is complete. As teachers grow more comfortable
with utilizing the program, I believe they will become more efficient in conducting their
assessments and they will begin to apply what they learn about student errors to their reading
instruction. Ultimately, both the students and teachers will benefit from the implementation of
this program.
As we conduct future evaluations of programs at Ivy Creek Elementary, it is my hope that
teachers will feel more and more comfortable sharing honest opinions with evaluators so that we
can fine tune the instructional practices that work best at the school.
EVALUATION REPORT 25
References
Al Otaiba, S., Hosp, J., Smartt, S., & Dole, J. (2008). The challenging role of a reading coach, a
cautionary tale. Journal of Educational & Psychological Consultation, 18(2), 124-155.
Retrieved from ERIC database.
Engstrom, M., & Danielson, L. (2006). Teachers' perceptions of an on-site staff development
model. Clearing House: A Journal of Educational Strategies, Issues and Ideas, 79(4),
170-173. Retrieved from ERIC database.
Fountas, I. & Pinnell, G. (2008). Assessment guide: A guide to the benchmark assessment
system. Portsmouth: Heinemann.
Hutchins, K., Arbaugh, F., Abell, S., Marra, R., & Lee, M. (2008). A consumer guide to
professional development. Science Scope, 31(8), 16-19. Retrieved from ERIC database.
Johnson, B., & Christensen, L. (2007). Educational research: Quantitative, qualitative, and
mixed approaches (3rd ed.). Boston: Allyn and Bacon.
Kaufman, R., Guerra, I., Platt, W. (2006). Practical evaluation for educators: finding what
works and what doesn’t. Thousand Oaks: Corwin Press.
Obara, S., & Sloan, M. (2010). Classroom experiences with new curriculum materials during the
implementation of performance standards in mathematics: A case study of teachers
coping with change. International Journal of Science and Mathematics Education, 8(2),
349-372. Retrieved from ERIC database.
Pinnell, G. & Fountas, I. (2007). The continuum of literacy learning: A guide to teaching.
Portsmouth: Heinemann.
Purdue University Online Writing Lab (OWL) (May 5, 2010). APA formatting and style guide.
Retrieved from http://owl.english.purdue.edu/owl/resource/560/01/.
EVALUATION REPORT 26
Appendix A
Survey – Fountas & Pinnell Benchmark Assessment Program
The following survey is designed to evaluate the Fountas & Pinnell Benchmark Assessment Program (Fountas & Pinnell Program.) It should take approximately 5-10 minutes to complete. Your answers will provide valuable information in determining the effectiveness of the program as it relates to reading instruction and professional development. No names will be used at any time when reporting data for this evaluation, so please answer honestly!
Background Information
1. How old are you?a. 20-29b. 30-39c. 40-49d. 50-59e. 60+
2. How long have you been teaching?a. 0-5 yearsb. 6-10 yearsc. 11-15 yearsd. 16-20 yearse. 21 or more years
3. What degree level do you currently hold?a. Bachelorsb. Mastersc. Specialistd. Doctoratee. Other, please specify
4. Have you used the Fountas & Pinnell Benchmark Assessment Program this year?a. Yesb. No
5. Have you previously used the Fountas & Pinnell Benchmark Assessment Program?a. Yesb. No
EVALUATION REPORT 27
Training and Professional Development
Please indicate how you strongly you agree or disagree with each of the following statements:
1 2 3 4 5Strongly Disagree
Strongly Agree
6. The one-hour pre-planning training session was beneficial to me.
1 2 3 4 5 N/A
7. The two 40-minute professional learning sessions during the first two weeks of school were
beneficial to me.
1 2 3 4 5 N/A
8. The one-on-one coaching provided by the reading coach/intervention coach was beneficial to
me.
1 2 3 4 5 N/A
9. The training DVDs were beneficial to me.
1 2 3 4 5 N/A
10. I received adequate training on implementing the Fountas & Pinnell Program.
1 2 3 4 5
11. I prefer receiving training at the beginning of the school year.
1 2 3 4 5
12. I prefer receiving training during the summer.
1 2 3 4 5
13. I prefer receiving training throughout the school year.
1 2 3 4 5
EVALUATION REPORT 28
14. Please add any additional comments about the nature of the Training and Professional
Development as they relate to the Fountas & Pinnell Program:
Implementation and Reading Instruction
Please indicate how you strongly you agree or disagree with each of the following
statements:
1 2 3 4 5Strongly Disagree
Strongly Agree
15. I find the Fountas & Pinnell Program to be user-friendly.
1 2 3 4 5
16. I believe that the Fountas & Pinnell Program accurately measures my students’ reading
level.
1 2 3 4 5
17. I use the Fountas & Pinnell Program to guide my reading instruction.
1 2 3 4 5
18. I have a greater understanding of student errors as a result of working with the Fountas &
Pinnell Program.
1 2 3 4 5
19. I use student errors as identified by the Fountas & Pinnell Program to guide my reading
instruction.
1 2 3 4 5
EVALUATION REPORT 29
20. The word lists assist me in my reading instruction.
1 2 3 4 5
21. The recording forms assist me in my reading instruction.
1 2 3 4 5
22. Having copies of the support documents in the teacher workroom is beneficial to me.
1 2 3 4 5
23. Please add any additional comments about the implementation and reading instruction as
they relate to the Fountas & Pinnell Program:
Fountas & Pinnell Program vs. Other Reading Assessment Programs
Please indicate how you strongly you agree or disagree with each of the following
statements:
1 2 3 4 5Strongly Disagree
Strongly Agree
24. The Fountas & Pinnell Program provides me with more information about my students
reading abilities than previous reading assessment programs.
1 2 3 4 5
25. The Fountas & Pinnell Program is more accurate than previous reading assessment
programs.
1 2 3 4 5
26. I prefer using the Fountas & Pinnell Program over previous reading assessment programs.
1 2 3 4 5
EVALUATION REPORT 30
27. Please add any additional comments about how the Fountas & Pinnell Program compares to
other reading assessment programs you have used:
Recommendation for Adoption
28. I would like to see the Fountas & Pinnell Program adopted for use at Ivy Creek Elementary
School.
1 2 3 4 5
29. I would like to see the Fountas & Pinnell Program adopted for use in Gwinnett County.
1 2 3 4 5
30. Please add any additional comments about the Fountas & Pinnell Program. Is there
anything you would like to see done differently next year?
If you would be willing to participate in a short interview session to further share your opinions
about the Fountas & Pinnell Program, please include your name: _______________________.
Again, all identifying characteristics of individual responses will remain confidential.
Thank you for taking time to complete this survey.
EVALUATION REPORT 31
Appendix B
Interview Guide – Fountas and Pinnell Benchmark Assessment Program
Tell me about your experience with using the Fountas and Pinnell Benchmark
Assessment Program.
What are the strengths of the Fountas and Pinnell Benchmark Assessment Program?
What are the weaknesses of the Fountas and Pinnell Benchmark Assessment Program?
What part of the training benefitted you the most?
Is there anything you feel should be done differently next year?
In what ways has the Fountas and Pinnell Benchmark Assessment Program impacted
your reading instruction?
In what ways has the Fountas and Pinnell Benchmark Assessment Program impacted
student achievement?
What kind of growth have you seen in your students’ reading levels?
How would you compare Fountas and Pinnell Benchmark Assessment Program to
programs you have used in the past?
What is the most valuable part of the Fountas and Pinnell Benchmark Assessment
Program?
What is the least valuable part of the Fountas and Pinnell Benchmark Assessment
Program?
Is there anything else you’d like to share?
EVALUATION REPORT 32
Appendix C
Language Impact Questionnaire
1. Which course of action would the evaluation report best support for 2011-12?
a. Continue the Fountas & Pinnell Benchmark Assessment Program in the same
way as 2010-11
b. Modify the Fountas & Pinnell Benchmark Assessment Program
c. Discontinue the Fountas & Pinnell Benchmark Assessment Program
2. Based on the evaluation report, how would you describe the effectiveness of the Fountas
& Pinnell Benchmark Assessment Program?
a. Completely successful – meets all program goals
b. Partially successful – meets some goals, achievement of remaining goals is yet to
be determined
c. Partially unsuccessful – meets some goals, remaining goals were not achieved and
will not be achieved with the current approach
d. Unsuccessful – none of the program goals have been achieved
3. In comparison to other reading programs, how would you rate the Fountas & Pinnell
Benchmark Assessment Program based on the evaluation report?
a. Clearly superior
b. Somewhat better
c. About the same
d. Less effective
e. Definitely worse
EVALUATION REPORT 33
4. According to the evaluation report, the main goals of the Fountas & Pinnell Benchmark
Assessment Program are…. (list as many as apply):
5. According to the evaluation report, the Fountas & Pinnell Benchmark Assessment
Program’s important strengths or accomplishments are…. (list as many as apply):
6. According to the evaluation report, the Fountas & Pinnell Benchmark Assessment
Program’s important limitations or deficiencies are…. (list as many as apply):
7. According to the evaluation report, the Fountas & Pinnell Benchmark Assessment
Program is perceived positively overall by what percentage of the users?
a. 90% or higher
b. 80%-89%
c. 70%-79%
d. 60%-69%
e. 50%-59%
f. 30%-49%
g. 0%-29%
EVALUATION REPORT 34
8. According to the evaluation report, which statement best describes the likelihood that the
Fountas & Pinnell Benchmark Assessment Program will positively impact Ivy Creek
Elementary School’s performance?
a. Virtually certain – positive impact is already evident
b. Probably – trends so far are positive
c. Possible – program design is sound, but no evidence is available at this time
d. Unlikely – trends so far are negative and the basic program design is flawed
e. Program will not positively impact performance. No impact or negative impact is
already evident.
Questionnaire based on Table 7.1, p. 211-12 (Kaufman, Guerra & Platt, 2006)
EVALUATION REPORT 35
Data Collection Organization Matrix
Questions Related Results
Level (Mega, Micro, Macro)
Required Data,
Measureable Indicators
Data Source
Data Collection Tools
& Procedur
es
Data Collected By (Date)
Primary Responsibili
ty
What impact do
professional learning sessions have on
teachers as they
implement the Fountas & Pinnell
Benchmark Assessment
System?
Teachers will be
confident and
successful reading
teachers.
MACRO
Teachers will
average a score of 4 or higher on the 5
point Likert scale
on professiona
l developme
nt questions.
Teachers will
express confidence
in implementi
ng the program.
Teachers will
indicate needs for additional
professional learning
with regards to
implementing the
program.
Teachers
Survey Results
Interviews
10/18/10
Kimberly Sharp
How does The MACR Teachers Teacher Survey 10/18/1 Kimberly
EVALUATION REPORT 36
the Fountas & Pinnell
Benchmark Assessment
System impact reading
instruction?
program will
positively impact reading
instruction.
O will average a score of 4 or higher on the 5
point Likert scale
on implement-ation and reading
instruction questions.
Teachers will
indicate that they analyze errors
made by students in
order to develop
appropriate reading
instruction methods.
Teachers will
indicate that
students show
growth in their
reading achievement and that
reading levels are positively impacted by the use
of the
s Results
Interviews
Informal: Student Reading Levels
0 Sharp
EVALUATION REPORT 37
program.How does
the Fountas & Pinnell
Benchmark Assessment
System compare to
the assessment
program used last
year?
Teachers will prefer
this program to previous
programs.
MACRO
Teachers will
average a score of 4 or higher on the 5
point Likert scale
on the Fountas & Pinnell vs.
other reading
programs questions.
Teachers will
express that the
new program is superior to previous
programs.
Teachers
Survey Results
Interviews
10/18/10
Kimberly Sharp
Should the Fountas &
Pinnell Benchmark Assessment System be
adopted for use at Ivy
Creek Elementary School next
year?
Teachers will
recommend that this
program be adopted at Ivy Creek
and in Gwinnett County.
MACRO
Teachers will
average a score of 4 or higher on the 5
point Likert scale
on the recommend-ation for adoption
questions...
Teachers
Survey Results
Interviews
Informal: Student Reading Levels
10/18/10
Kimberly Sharp
Matrix taken from Figure 4.3, p. 105 (Kaufman, Guerra, & Platt, 2006).