The Educare Chicago Research-Program Partnershipand Follow-Up Study: Using Data on Program Graduates
to Enhance Quality Improvement Efforts
Amanda Stein, Karen Freel, Ann T. Hanson, and Debra Pacchiano
The Ounce of Prevention Fund Chicago, Illinois
Brenda Eiland-Williford
Educare Chicago, Chicago, Illinois
Early childhood education programs that have succeeded in raising the quality and outcomes of their
programs have done so, oftentimes, by championing a culture of continuous improvement. This
includes comprehensive formative evaluation and routines of data feedback and discussion that
support teaching staff to use child growth and classroom quality data to inform daily instructional
decisions and interactions with families. This article provides a case illustration of how leaders
and staff at 1 high-quality early childhood education program (Educare Chicago) use data and feed-
back from graduate children and families to inform and drive continuous quality improvement
through a Research-Program Partnership, which is a dynamic interchange between researchers,
program leadership, and practitioners. Research Findings: This article focuses on the utilization
of longitudinal Follow-Up Study data obtained from Educare Chicago children and families as they
transitioned into the elementary school system to inform early childhood program quality enhance-
ment efforts. Quantitative data on school type, school quality, and teacher ratings of children and
parents are included. The utilization of findings from qualitative data obtained from parents and
Educare program staff are also discussed. Practice or Policy: Follow-Up Study key findings and
program responses are described, in addition to the role of the Research-Program Partnership in
fostering both these exchanges and a deeper understanding of how to use data from program
graduates and their families. Implications for program practices specific to children and families’
transition to kindergarten, early mathematics instruction, and family engagement are discussed.
In the fields of early learning and education, the demand for the utilization of research-based
practices, data-driven decision making, and accountability is clear. These expectations not only
permeate the dialogue and debates of policymakers, administrators, and even frontline practi-
tioners but also pervade federal and state policies and the regulations and standards that provide
guidance on carrying out such mandates. For example, the No Child Left Behind Act of 2001
(PL 107-110) and the Improving Head Start for School Readiness Act of 2007 (PL 110–134)
both require the implementation of curricula, policies, practices, observational instruments,
Correspondence regarding this article should be addressed to Amanda Stein, The Ounce of Prevention Fund, 4859
South Wabash Avenue, 2nd Floor, Chicago, IL 60615. E-mail: [email protected]
Early Education and Development, 24: 19–41
Copyright # 2013 Taylor & Francis Group, LLC
ISSN: 1040-9289 print/1556-6935 online
DOI: 10.1080/10409289.2013.739542
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
and other strategies that have demonstrated effectiveness stemming from research. The Obama
Administration has declaratively pronounced program evaluation as a priority across agencies to
determine whether ‘‘programs are achieving their intended outcomes as well as possible and at
the lowest possible cost’’ and to further ‘‘strengthen the design and operation of programs’’
(Orszag, 2009, para. 1). In general, there is a growing expectation that early childhood programs
collect and report child-level results as a basis for individualizing services and documenting their
program-wide impact and improvement processes, in addition to using such data at all levels to
guide decision making (Snow & Van Hemel, 2008; U.S. Department of Education and U.S.
Department of Health and Human Services, 2011).
WHAT COUNTS AS RESEARCH, EVIDENCE, OR DATA IN EARLYCHILDHOOD PROGRAMS?
The demand for data and evaluation is unprecedented, yet research indicates that education
stakeholders disagree on how evidence or research should be defined, acquired, interpreted,
and effectively utilized (Tseng, 2012). For example, in their review of 52 studies, Honig and
Coburn (2008) examined the ways in which school district staff responded to federal and state
policies mandating the use of research and data. They found that district personnel defined
‘‘evidence’’ as inclusive of social science research, student and school achievement data,
program evaluation findings, practitioner knowledge, expert opinions, and parent and com-
munity input. Yet other researchers have found that education leaders and policymakers most
readily associate ‘‘scientifically based research’’ with gold-standard randomized controlled trials
(Nelson, Leffler, & Hansen, 2009). A joint position statement from the National Association for
the Education of Young Children (NAEYC) and the National Association of Early Childhood
Specialists in State Departments of Education (NAECS=SDE) indicated that this view is too
narrow, stating that sources of usable evaluation data in early childhood should include child
or family progress monitoring measures or assessments, administrator ratings, program self-
assessments, implementation checklists, observational measures of practice, staff input, and
constituent feedback (NAEYC and NAECS=SDE, 2003).
WHY USE RESEARCH, EVIDENCE, OR DATA IN EARLY CHILDHOODPROGRAMS, AND HOW IS IT DONE?
According to Fuchs, Mock, Morgan, and Young (2003), data-based decision-making or
problem-solving approaches in education are grounded in the principle that ‘‘no student character-
istic (e.g., disability label, race, [socioeconomic status], neighborhood) dictates a priori what inter-
vention will work. Nor will a given intervention be effective for all students of a particular group’’
(p. 160). A variety of models exist to guide early childhood educators and administrators in this
effort to implement data-driven decision making at the individual child level (e.g., Coleman,
Buysse, & Neitzel, 2006; Gresham, 2007) and at the program, district, and state levels (e.g., Rous,
McCormick, Gooden, & Townley, 2007; Stedron, 2009). Even with these models, early childhood
educators tend to rely on intuition and informal observations rather than the collection and
systematic use of data to guide decision making (Sandall, Schwartz, & LaCroix, 2004).
20 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
Furthermore, research has shown that even when assessment data are available, early child-
hood professionals often lack the preparation necessary to effectively interpret and use it (Horton
& Bowman, 2002; Scott-Little, Kagan, & Clifford, 2003; Stiggins, 1999). Yet optimal outcomes
for children and families will not be realized unless early childhood leaders and staff have the
capacity to understand and implement effective data utilization practices, including selecting
the appropriate type and format of data, to inform routine decision making about classroom
practice, work with families, professional development, and environmental and program
supports (Fuligni, Howes, Lara-Cinisomo, & Karoly, 2009; Kagan, 2003; Kincaid, Childs,
Blase, & Wallace, 2007; Schwartz & Olswang, 1996).
It is therefore critical to define the conditions and processes that facilitate the collection,
understanding, and use of formative assessment and evaluation data to make data-driven quality
improvement a consistent component of early childhood practice. This article provides a descrip-
tion of how one program, a national model of high-quality early childhood education emphasiz-
ing research-based practice and functional program evaluation, used data and feedback from
graduate children and families to inform and drive program change.
THE EDUCARE SCHOOL MODEL
In 2000, the Ounce of Prevention Fund opened the first Educare School on Chicago’s south side
to prepare vulnerable children and their families for success in school and in life. Now a national
network of 17 schools in diverse communities, Educare provides high-quality, full-day and
full-year early childhood education for children from birth to 5 years old (for more information,
see www.educareschools.org). The primary aim of each school is to prevent the achievement gap
between children in poverty and their more advantaged peers that is evident long before kinder-
garten entry (Halle et al., 2009; Hart & Risley, 1995; National Center for Education Statistics,
2001). Educare leaders therefore have a strategic focus on children’s achievement and organize
their systems, programmatic approaches, and professional development to support effective
teaching and family engagement. Each Educare School meets Early Head Start and Head Start
program performance standards and also implements a comprehensive set of evidence-based
core components that intentionally support high-quality teaching and family engagement (see
Table 1). Ongoing, independent evaluation of the nationwide network of Educare Schools
demonstrates that children in poverty who participate in the program, including dual language
learners, enter kindergarten ready for school, achieving near national norms on measures of
vocabulary, social-emotional development, and school readiness and well above typical scores
for children at risk of school failure (West, Malone, Hulsey, Aikens, & Tarullo, 2010; Yazejian
& Bryant, 2012).
EDUCARE CHICAGO RESEARCH-PROGRAM PARTNERSHIP (RPP)
Most relevant to the current article is that each Educare School—as part of the core feature of
using research-based practices and strategies (see Table 1)—establishes a local RPP. The
original design of Educare’s RPP was informed by the local evaluation partnerships developed
for the Early Head Start Impact Study (Love et al., 2005) and the principles of utilization-
focused evaluation (Patton, 2008) and participatory evaluation (Ryan & DeStefano, 2000).
USING DATA TO ENHANCE QUALITY IMPROVEMENT 21
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
Although there is variation in the structure and processes of local RPPs, each RPP connects
interdisciplinary program leadership and staff with a PhD-level Local Evaluation Partner, and
they together implement a dynamic, reciprocal data feedback and utilization cycle. Each RPP
supports routine, frequent examination of systems, practices, and attainment of goals for
program monitoring and continuous quality improvement at the child, family, program, and
agency levels (see Figure 1).
TABLE 1
Core Features of the Educare School Model
1. Provide full-day, full-year services.
2. Maintain high staff-to-child ratios and small class sizes.
3. Use research-based practices and strategies.
4. Provide continuity of care beginning at birth to help children develop secure relationships.
5. Provide onsite family engagement and support services to families.
6. Maintain high staff qualifications and embedded, intensive professional development.
7. Emphasize social-emotional development to promote positive approaches to learning and school readiness.
8. Provide an enhanced focus on early oral language and literacy.
9. Provide an enhanced focus on problem solving and numeracy.
10. Integrate the arts.
11. Start early, with an emphasis on prenatal services.
12. Implement an interdisciplinary approach.
13. Implement reflective supervision and practice throughout the program.
FIGURE 1 Educare Chicago Research-Program Partnership data feedback and utilization cycle.
22 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
RPP Membership and Roles
At Educare Chicago, the Local Evaluation Partner and the school’s Director of Program and
Curricula colead the RPP, together championing the importance of data gathering and utilization
for continuous improvement. Supervisors—including master teachers, curriculum specialists,
and family support leaders—also participate in the Chicago RPP, playing a critical role in the
dialogue and utilization of data to create and implement quality improvements across the
Educare School. In particular, these program leaders supervise, coach, and support the classroom
teaching teams and family support specialists who directly implement the program and educate
children and families.
RPP Data Feedback and Utilization Process
As noted in Figure 1, the RPP implements Educare’s data feedback and utilization cycle, includ-
ing monitoring data collection and assessment; discussing available data and identifying new
analyses or questions; and addressing implications for intervention planning, goal setting, and
staff and program development. Based on research noting the importance of providing data ‘‘with
sufficient frequency and in contexts that will inspire educators to develop, utilize, and disseminate
effective interventions’’ (Downs & Strand, 2006, p. 678), the RPP utilizes data from a variety of
quantitative and qualitative sources (e.g., standardized measures to compare children’s skills to
national norms, curriculum-based assessment, progress-monitoring tools, teacher=parent ratings,interviews, focus groups) to provide a fuller picture of program functioning and impact
(Scott-Little et al., 2003).
The relationships forged and knowledge developed through the Educare Chicago RPP pro-
vide a unique opportunity to monitor the continued progress of children and families after they
graduate from Educare. The following case example moves beyond the analysis of current
program data and focuses on how data about child and family experiences and outcomes during
the transition to kindergarten and through the third grade have led to change in Educare Chicago
program practices and policies.
CASE EXAMPLE: EDUCARE CHICAGO ELEMENTARY SCHOOLFOLLOW-UP STUDY
A unique feature of the Educare Chicago RPP is the collection of data to track the progress of
Educare children and families as they transition to kindergarten and advance through the primary
grades. This ongoing Educare Chicago Elementary School Follow-Up Study is a case illustration
of how researchers, practitioners, and school leaders can work collaboratively to use data for
program quality improvements. In the current article, we highlight aspects of the design,
implementation, and impact of the Educare Chicago Elementary School Follow-Up Study,
emphasizing how this unique, ongoing study has enhanced the effectiveness of early learning
experiences in Chicago’s Educare School.
The Educare Chicago Elementary School Follow-Up Study (henceforth, the Follow-Up
Study) was conceived as, and continues to be, an exploratory, descriptive study aimed at
examining the experiences and outcomes of children and families after they graduate from the
USING DATA TO ENHANCE QUALITY IMPROVEMENT 23
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
Educare Chicago program. An interest in following children into elementary school has long
been shared by Educare leadership and executives, board members, and staff from the Ounce
of Prevention Fund. In particular, these stakeholders were interested in whether the social-
emotional and academic gains children made during their enrollment at Educare were sustained
and grew through their primary school years.
With the research knowledge that gains made in early learning are more likely sustained when
children experience higher quality primary school environments and instruction (Reynolds,
2003), members of the Educare Chicago RPP team (which includes us) were also eager to learn
more about the kindergarten and primary grade expectations and environments that Educare
graduates experienced and how well parents were supported in partnering with primary grade
educators. Thus, the three main goals of this study were (a) to examine how effectively Educare
Chicago prepared young children and their families from low-income backgrounds for school
success, (b) to learn about the school environments and expectations of the primary grade
teachers, and (c) to discover implications from these findings to inform program improvement
decisions.
Based on these collaboratively developed aims, we designed the Follow-Up Study to explore
the following issues and questions:
1. Given the vast range of both school options and quality in Chicago’s urban public
school landscape, what schools did Educare parents explore for enrollment, and what
elementary schools did children attend? Information on the number of schools, types
of schools (e.g., neighborhood, charter, magnet, private), and quality of schools (i.e.,
the Illinois State Board of Education publishes an annual School Report Card with
quality indicators, including annual yearly progress percentages) was known anecdo-
tally, but there had been no systematic collection or analysis of this information prior
to the formalization of the Follow-Up Study.
2. How did children and their parents experience the transition into kindergarten?
Although the RPP team and Educare staff members knew that the kindergarten class-
room was likely to be quite different from an Educare School early learning class-
room, we did not know how well children would adjust or what difficulties
children might have in their adjustment to the new environment. Similarly, we sought
to understand how well parents adjusted to the elementary school environment and
whether they would continue to initiate communication with their child’s educators—a practice emphasized at Educare. In addition, we knew that parents would have
insights helpful to Educare’s program improvement efforts that would be based on
their perceptions of their child’s transition to kindergarten and what the kindergarten
teacher was communicating about their child’s readiness for school.
3. How prepared for kindergarten were Educare Chicago graduates academically, socio-
emotionally, and behaviorally? Prior to the Follow-Up Study, contact with parents
once their children transitioned to kindergarten was sporadic and infrequent. Even
when parents still had younger children enrolled in the Educare program, conversa-
tions about the older child who had graduated Educare were informal and not
recorded. Program staff therefore had no systematic information about the academic
achievement, social-emotional competence, or behavior of Educare Chicago
graduates in elementary school.
24 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
4. How prepared for kindergarten and elementary school were Educare Chicago
parents? Did parents continue to be involved in supporting their child’s learning
and development? Did parents continue to engage with their child’s elementary
school and communicate with educators about the child’s progress? Even with some
contact with parents of Educare Chicago graduates, it was difficult to gauge whether
parents retained and put into practice the value of remaining engaged with supporting
their child’s learning, staying involved in their child’s education, and actively
communicating with educators and other school staff to support their child’s school
success.
METHOD
To answer these questions and provide a rich description of the issues, we designed the Follow-Up
Study using a combination of quantitative and qualitative data. Quantitative data included teacher
surveys that asked for ratings of a child’s readiness at grade entry and exit as well as level of parent
participation, grades, type of school attended, and standardized assessments. Qualitative data
included parent interviews and focus groups. We did not build a comparison=control group into
this exploratory study. The primary purpose of this study was not to contrast the elementary school
progress and achievement of Educare graduates with that of children who had not attended
Educare Chicago but rather to examine issues related to school readiness and the transition from
early education to the K–12 school system, to follow the course of Educare children’s achieve-
ment, and to learn from parents about their perceptions and experiences of their child’s school
readiness and continued efforts to support their child’s learning and development.
Participants
The study population consisted of children who attended and transitioned into kindergarten from
Educare Chicago starting in the fall of 2005 and their parent(s). Each year thereafter, children
and families were recruited for participation in the Follow-Up Study at the time of their
transition to kindergarten and were then followed longitudinally with data collected on an annual
basis. The first cohort of graduates that transitioned to kindergarten in the fall of 2005 included
45 children, 34 of whose parents consented to participate in the Follow-Up Study. Although
cohorts of children and families continue to be added to the Follow-Up Study sample as they
transition out of Educare Chicago, this article focuses on six cohorts of children whose parents
consented to their participation and provided data (n¼ 172). In Spring 2011, the sixth cohort of
children finished their kindergarten year and the first cohort of children finished fifth grade in
elementary school. The second column of Table 2 provides the initial cohort sample size,
defined as the number of children whose parents consented to participate in the Follow-Up Study
during their transition or last year in Educare Chicago. The third column lists the recruitment
percentage, which was calculated by dividing the number of children whose parents consented
by the total number of Educare children transitioning to kindergarten in that particular year.
Throughout the study, the research team worked with Educare family support staff to obtain
up-to-date contact information on participating families, which resulted in the relatively high
retention rate of 74% or more as shown in the fourth column of Table 2. Contact with families
USING DATA TO ENHANCE QUALITY IMPROVEMENT 25
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
in the Follow-Up Study is annual (at a minimum), with the level of participation fluctuating
within a cohort from year to year.
Educare Chicago is also an Early Head Start and Head Start program; therefore, all families
eligible for enrollment were at or below the federal poverty level for each year of program
participation. Educare Chicago is located in a primarily African American community, and all
children in this Follow-Up Study sample were African American (with the exception of one
child of Hispanic ethnicity) and spoke English as their primary language. Table 3 details the
demographic information on the Follow-Up Study sample of children and families from Cohorts
1 through 6. We collected these data while children were enrolled in Educare. The data include
only those children whose parents provided consent for participation in the study. We did not
collect archival data on children whose parents did not consent to participate in the Follow-Up
Study.
Design and Procedures
Table 4 presents the study design and data collection schedule. Additional details are provided
here.
Kindergarten through second grade. During the kindergarten, first-, and second-grade
years, we individually interviewed the parents once in the fall=winter. In addition, we asked chil-dren’s teachers to fill out a short survey about the child’s and family’s preparedness for the cur-
rent grade; the child’s and family’s readiness for the following school year; the child’s general
level of functioning in literacy, math, and social skills; as well as information about parent
involvement. We collected information about grades, special education status (i.e., whether
the child had an individualized education program), retention, and attendance from the school
district and via school report cards.
Third grade. At the end of the child’s third-grade year, we asked parents to come to the
study offices with their child. We typically scheduled between three and five parents and
children to arrive at the same time. During this visit, research staff members administered the
standardized assessments listed in Table 4, which had previously been administered with each
TABLE 2
Educare Chicago Elementary School Follow-Up Study Cohort Year, Sample Size, Recruitment
Percentage, and Participation Retention Rate
Cohort number
(Kindergarten Year)
Initial sample size for educare
transition year (N)
Recruitment
percentage
Participant retention
rate (%)
Cohort 1 (2005–2006) 34 76 79
Cohort 2 (2006–2007) 24 73 74
Cohort 3 (2007–2008) 31 84 90
Cohort 4 (2008–2009) 26 68 100
Cohort 5 (2009–2010) 31 78 97
Cohort 6 (2010–2011) 26 76 100
26 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
child soon before he or she left Educare for kindergarten. While the children were being indi-
vidually assessed, other research staff members conducted a focus group with the parents and
also asked parents to complete a rating of their child’s social-emotional competencies. We asked
third-grade teachers to fill out the annual teacher survey and complete a rating of students’ social
skills identical to the survey completed by parents. We then obtained school performance data,
including scores on the state-required Illinois Standards Achievement Test. In the fourth, fifth,
and subsequent grades (upcoming), we obtained school performance data only. Because of
resource and budget limitations, we did not continue interviews with parents, direct child
assessments, and teacher surveys and ratings beyond the third-grade year.
Measures
Parent interviews and focus groups. The parent interviews and focus groups used a semi-
structured interview guide that focused on parents’ perceptions in a variety of areas, including
their child’s transition and adjustment to kindergarten, experiences with their child’s elementary
school and teachers, involvement in school activities, ways in which they supported their child’s
education and learning, how well they thought their child was prepared for school, and what
TABLE 3
Educare Chicago Elementary School Follow-Up Study Child and Family Demographics
Maternal education level (%)
Cohortnumber
(Kindergarten
Year)
Child
gender
(%male)
Children
on
an IEP
Maternal
age at child birth
(M,range)
Less than
high school
High school or
equivalent
Some
college
Bachelor’s
degreeor more
Cohort1
(2005–
2006)
36 0 25,13–43 11 11 78 0
Cohort2
(2006–
2007)
42 0 21,15–33 15 15 54 15
Cohort3
(2007–
2008)
42 3 22,10–37 8 8 44 40
Cohort4
(2008–
2009)
45 6 23,12–40 3 17 59 21
Cohort5
(2009–
2010)
24 7 23,14–38 6 15 67 12
Cohort6
(2010–
2011)
31 7 25,15–48 0 7 77 16
Note. IEP¼ individualized education program.
USING DATA TO ENHANCE QUALITY IMPROVEMENT 27
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
thoughts they had about how Educare Chicago could better prepare its graduates and their fam-
ilies for elementary school and the transition into kindergarten. We conducted one-on-one parent
interviews annually from kindergarten through second grade. We structured parent focus groups
TABLE 4
Educare Chicago Elementary School Follow-Up Study Measures and Timeline
Measure Construct Source Timing
Standardized assessments of student achievement
Devereux Student
Strengths Assessment
Rating of student’s
social-emotional
competence and
behavior
Parent and teacher Spring of third gradea
Peabody Picture
Vocabulary Test–4
Receptive vocabulary and
concept development
Reliable assessor Spring of third gradeb
Woodcock–Johnson III
Tests of Achievement:
Applied Problems
Early math skills
development
Reliable assessor Spring of third gradeb
Illinois Standards
Achievement Test
Illinois standardized
test—math and reading
beginning in the third
grade and each year
thereafter
School Annually
Other measures
Individual parent
interview
Child’s preparedness for
kindergarten, transition
to kindergarten;
parent’s focus on and
engagement with
child’s learning, parent–
child relationship
Parent Annually, kindergarten
through second grade
FFocus group Parent’s focus on and
engagement with
child’s learning, parent–
child relationship
Parent Spring=summer of third
grade
Teacher survey Child’s readiness for
current and upcoming
grade in school, special
education identification,
grade retention, parental
involvement
Teacher Annually, kindergarten
through third grade
School performance data Grades, IEP status,
retention status,
attendance
School Annually
Note. IEP¼ individualized education program.aIn the fall and spring prior to kindergarten entry, Educare teachers completed the Devereux Early Childhood Assess-
ment (a preschool companion measure to the Devereux Student Strengths Assessment).bThis measure was also completed in the fall and spring prior to kindergarten entry, before the child transitioned from
Educare Chicago.
28 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
(third grade) to be in small groups to allow parents to dialogue about their responses to questions
and experiences.
During the first year of interviews, parents expressed that they also wanted to tell us about
significant events in their lives—some positive and some negative. Recognizing that these
events had a bearing on parents’ abilities to focus on their children’s education and on child out-
comes, we added questions to the interview protocol that invited parents to tell us about their
lives, about specific events that might have occurred in the past year, and about whether these
events affected their ability to support their children in school.
Teacher survey. We adapted the Teacher Survey from the Chicago Child-Parent Center
Longitudinal Study (Reynolds, Bezruczko, & Hagemann, 1997) for use in the Educare Chicago
Elementary School Follow-Up Study. The annual Teacher Survey was used to collect teacher
ratings of children’s school readiness overall and in specific academic areas, for the current
grade and the next grade.
School administrative data. We collected child grades, special education status, retention,
attendance, and Illinois Standards Achievement Test scores at all grade levels directly from the
child’s school=teacher or parent (e.g., usually from a report card) or via a data request to the
primary school district serving Educare Chicago graduates, which was most often Chicago Pub-
lic Schools.
Direct child assessments. Mirroring the assessments given in the spring prior to Educare
students transitioning to Kindergarten, research staff directly assessed children at the end of
third grade with the Peabody Picture Vocabulary Test–4 (Dunn & Dunn, 2007) and the
Woodcock–Johnson III Tests of Achievement: Applied Problems (Woodcock, McGrew, &
Mather, 2001). Parents and teachers completed the Devereux Student Strengths Assessment
for children (LeBuffe, Shapiro, & Naglieri, 2009).
RESULTS AND PROGRAM RESPONSE
This section is organized by the four major research questions. For each research question, we
present findings followed by a discussion of the resulting Educare program response or quality
improvement effort. The Educare Chicago RPP team partnered with and supported school lead-
ership and staff in generating these program responses through its data feedback and utilization
process, which included summarizing, analyzing, and reviewing both current program data and
Follow-Up Study data; discussing the implications and meaning of the data; generating a plan
for modifications or program quality improvements; and monitoring and evaluating implemen-
tation. The cohorts with available data and the sample size for each type of data collected are
reflected in the findings discussed here.
What Elementary Schools Did Educare Children Attend?
Key findings. Analysis of the first cohort of children in the Follow-Up Study highlighted
several characteristics of the elementary schools that Educare Chicago graduates attended. Most
notably, 45% of the 29 different schools that Educare children attended were on the State of
USING DATA TO ENHANCE QUALITY IMPROVEMENT 29
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
Illinois’s Academic Watch List, with 57% of Educare graduates attending these low-performing
elementary schools. We were able to use several proxy variables to obtain data about the quality
of schools attended by Educare Chicago graduates. Table 5 compares these Chicago Public
Schools neighborhood schools to charter and magnet schools attended by the first five cohorts
of Educare Chicago graduates and illustrates how charter and magnet schools surpassed neigh-
borhood schools on every quality indicator. It is important to note that Table 5 also shows that
the annual yearly progress for the neighborhood schools that Educare Chicago graduates
attended was quite poor at just 4%. Reynolds, Ou, and Topitzes (2004) found that the most
significant factor predicting later positive educational attainment and lower juvenile arrest was
attendance at a high-quality elementary school following participation in preschool.
Program response. Educare program staff significantly bolstered their transition activities
and efforts with parents before the second cohort of children transitioned into kindergarten in
order to increase the number of children attending high-quality elementary schools. In ensuing
years, the transition into kindergarten became a year-long process that included several connec-
ted events scheduled throughout the school year. These new transition activities were based on
the quantitative Follow-Up Study findings described previously, as well as qualitative data from
parent interviews and on-the-ground knowledge of parent experiences from Educare program
staff. New transition activities included the following:
. contact with charter and magnet schools in the area, with a focus on those attended by
previous Educare graduates, regarding specific directions on application and enrollment
processes so that parents had multiple options to select appropriate, higher quality
schools for their child to attend;
. several transition events with parents that explained the array of school types available to
them, provided information on the schools parents were considering, helped parents
TABLE 5
Comparison of School Quality Indicators for Schools Attended by Educare Chicago Graduates
School quality indicator
CPS neighborhood schools
(N¼ 25)
CPS charter and magnet schools
(N¼ 14)
Attendance (%) 93.2 95.1
Povertya(%) 96.6 77.3
Student mobility rateb 30 13.9
Special educationa(%) 12.1 10.8
Meeting adequate yearly progress (%) 4% (1 out of 25 schools) 50% (7 out of 14 schools)
Meets ISAT in readingc(%) 50.1 74
Meets ISAT in mathc(%) 67.8 85.5
Note. ISAT¼ Illinois Standards Achievement Test.aThe percentage of students living in poverty and the percentage of students receiving special education services are
important because those students generally have greater educational and supportive needs that may require additional
resources.bThe student mobility rate is based on the number of students who enroll in or leave a school during that particular
school year. In schools with high mobility rates, all children may be affected because of continual readjustments of school
staff to accommodate the influx of new students.cThe ISAT measures elementary school students’ performance against the Illinois Learning Standards using both
multiple choice and open-ended items. Starting in third grade, students complete the Reading and Math ISAT annually.
30 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
identify high-quality elementary schools that best met their child’s and family’s needs,
provided comparisons between schools based on indicators of quality data, and assisted
parents with application and enrollment forms for their chosen school options; and
. an annual special event for parents whose children would be transitioning to kindergar-
ten in the fall to have the opportunity to engage with kindergarten teachers, other
elementary school leaders and staff, and already-transitioned families (specifically, fam-
ilies in the Follow-Up Study) to hear about kindergarten learning expectations, get tips
on how parents can support their children during this transition, and hear ideas for sus-
taining learning at home.
Follow-Up Study data from subsequent cohorts indicated that a decreased number of parents
enrolled their children in local Chicago Public Schools neighborhood schools and an increased
number of families enrolled their children in higher quality schools such as magnet, charter, or
gifted center schools. Table 6 shows the change in school choice and enrollment across the six
cohorts.
How Did Educare Children and Their Parents Experience the Transition IntoKindergarten?
Key findings. Parents in the Follow-Up Study highlighted how their children transitioned
from Educare with excitement for kindergarten, a positive attitude toward learning, and the
ability to get along with classmates. They also reported that Educare Chicago was instrumental
in ensuring that their child was prepared for kindergarten socioemotionally, behaviorally, and
academically. However, the interviews also uncovered many of the challenges families and
children face in adjusting to a new school environment. Once the kindergarten year was in full
swing, parents and children experienced a stressful and long-term adjustment period stemming
from the contrasts between the kindergarten academic environment and the Educare child- and
family-centered environment. Specifically, parents reported that the kindergarten classroom
TABLE 6
Types of Elementary Schools in Which Educare Graduates Are Enrolled and Attending Kindergarten
School type
Cohort 1
(N¼ 32)
Cohort 2
(N¼ 26)
Cohort 3
(N¼ 33)
Cohort 4
(N¼ 32)
Cohort 5
(N¼ 28)
Cohort 6
(N¼ 26)
Total
(N¼ 177)
CPS neighborhood
school
21 (66%) 14 (54%) 16 (47%) 18 (56%) 11 (39%) 10 (38%) 90 (51%)
CPS charter, contract
school
3 (9%) 7 (27%) 11 (33%) 8 (25%) 9 (32%) 15 (58%) 53 (30%)
CPS magnet, gifted,
classical school
5 (16%) 5 (19%) 5 (17%) 6 (19%) 6 (21%) 1 (4%) 28 (16%)
Private, religious,
non-CPS school
3 (9%) 0 (0%) 1 (3%) 0 (0%) 2 (7%) 0 (0%) 5 (3%)
Note. CPS¼Chicago Public Schools.
USING DATA TO ENHANCE QUALITY IMPROVEMENT 31
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
routine required substantial self-direction and that teachers had extremely high expectations for
independent learning and sustained focus.
Program response. The alignment of preschool to kindergarten and through third grade
has enjoyed a recent focus in research and policy (Bogard & Takanishi, 2005; Reynolds,
2003; Reynolds & Temple, 2008). Such alignment has the potential to mitigate the wrenching
transitions parents described (Reynolds, Magnuson, & Ou, 2010) and has been associated with
better student outcomes (Marietta, 2010). Alignment efforts in which the early childhood
programs and elementary schools are not co-located, such as in the Child-Parent Centers
(Reynolds, 2003), requires dedicated time between willing and committed partners (e.g., see
‘‘A Cohesive Educational Experience and Other Challenging Issues,’’ regarding efforts
between Educare Chicago and two charter elementary schools). Such an alignment with the
many Chicago public elementary schools attended by Educare graduates was unfeasible; thus,
the program response focused on preparing parents for the transition and incorporating
age-appropriate methods in the classroom that would better prepare children for the elementary
school environment. Educare Chicago staff used the key findings from the Follow-Up Study
to create more focused and useful transition materials, workshops, activities, and experiences
for children and parents. New transition strategies in the year prior to kindergarten included
the following:
. explaining to parents the differences between kindergarten’s more academically centered
environment and Educare’s more child-centered environment;
. describing to parents the demands and stressors these differences will place on
their child (e.g., increased levels of self-direction, independence, and sustained
attention);
. highlighting specific ways in which parents can interact with and support their child both
directly and through partnership and regular communication with the child’s teacher and
other school staff;
. enhancing Educare classroom instruction and activities to better promote children’s
abilities to sustain attention, focus on detail, and gain exposure to and practice with
the types of expectations of their future kindergarten classroom (e.g., independent
learning and work); and
. offering exposure to and practice with the similarities and differences between the
child’s current classroom and future kindergarten classroom.
Through this process of examining data to develop processes that facilitate children’s and
families’ transitions to kindergarten, Educare leadership identified the need to seek out opportu-
nities to engage in dialogue and learn from leaders and staff at nearby elementary schools to
better understand the school’s educational programming and family engagement practices, thus
continuing to build the knowledge necessary to design the most helpful transition supports. Such
transition supports offered as a result of data from the Follow-Up Study have been shown to
build long-term patterns of parent involvement (Kraft-Sayre & Pianta, 2000) and have been
associated with higher academic achievement at the end of the kindergarten year (Schulting,
Malone, & Dodge, 2005). The RPP team and Educare staff continue to consider strategies to
incorporate this important and substantive feedback in age-appropriate and family friendly ways
at Educare.
32 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
How Prepared for Kindergarten Were Educare Graduates Both Academically andBehaviorally?
Key findings. Data from the first two cohorts’ kindergarten grades indicated that math pro-
ficiency was not as high as literacy proficiency (see Table 7). Although kindergarten teachers
rated Educare Cohorts 1 and 2 as prepared ‘‘overall’’ on the teacher survey, the proficiency
scores (generated from grades on report cards) told a slightly different story. Table 7 reflects
the literacy and math proficiency levels in kindergarten for Cohorts 1 and 2. Parent interviews
reinforced this early finding, in that parents reported that children struggled more with math in
kindergarten than with other subjects.
Program response. A meta-analytic study by Duncan et al. (2007) showed that early math
skills had the greatest predictive power for later academic success, followed by reading and then
attention skills. Furthermore, a report for the National Research Council (Cross, Woods, &
Schweingruber, 2009) on early math concluded that although all young children have the
capability to become competent in mathematics, that potential was not realized because oppor-
tunities to learn mathematics were lacking in early childhood settings, homes, and communities.
It was clear that a program response to math learning was needed, equal to efforts being made to
boost and strengthen literacy instruction and outcomes. Furthermore, the Follow-Up Study
demonstrated that the math proficiency levels in kindergarten for the first two cohorts reinforced
what the Educare classroom teachers and leadership staff were discovering from their classroom
curriculum-based assessment data: the need to launch a new initiative to increase math achieve-
ment. The resulting multiyear Early Math Initiative focused on achieving this through family
engagement and professional development. The family engagement component included (a)
helping parents understand the importance of early math, (b) working with parents to incorporate
math learning into their home activities to enhance classroom math learning, and (c) providing
parents with guided take-home math activities.
The Early Math Initiative’s professional development component included a partnership with
the Erikson Institute, a leading graduate school in early childhood education, to implement a set
TABLE 7
Kindergarten Literacy and Math Proficiency Levels for Follow-Up Study Cohorts 1 and 2
Subject areaGrades indicating meeting or exceeding
proficiency (%)Grades indicating approaching or below
proficiency (%)
Cohort 1 (N¼ 22)
Literacy 54 46
Math 27 73
Cohort 2 (N¼ 18)
Literacy 67 33
Math 50 50
Note. As there are different grading methods at different elementary schools, Follow-Up Study research staff
developed a coding system for grades that resulted in the categories of exceeding, meeting, approaching, and below
proficiency.
USING DATA TO ENHANCE QUALITY IMPROVEMENT 33
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
of activities focused on strengthening the mathematical pedagogical content knowledge and
math instruction skills of Educare teaching staff. Activities included the following:
. engaging in an intensive, year-long training of preschool teachers and family support
staff on core math concepts;
. using core math concepts in lesson planning;
. monitoring children’s progress in mathematical knowledge and skills;
. providing coaching support for teachers on the use of new math content knowledge,
instructional skills, and the use of children’s progress data to plan and execute
high-quality math lessons across teaching formats (e.g., individual, small group, large
group); and
. collecting and analyzing the current research literature to conceptually extend core math
concepts for children younger than 3, including pilot teaching and coaching of birth-to-3
math core concepts in infant and toddler classrooms.
The Early Math Initiative continues with intensive staff training, coaching of preschool
teachers, and the development of the birth-to-3 core math concepts as well as math materials
and activities for families.
How Prepared for Kindergarten and Elementary School Were Parents? And, DidParents Continue to Be Involved with Their Child’s Elementary School and withCommunicating with Educators about Their Child’s Progress?
Key findings. A substantial majority of Educare parents were consistently rated—across
time and grade levels—as supportive of their child’s learning and involved in their child’s learn-
ing at school, as documented by teacher survey ratings of parent involvement. Teacher ratings
indicated that the majority of Educare parents participated in activities at the school, returned
forms and communications, picked up their child’s report card, and initiated a visit to the school
and conversation with the teacher. What is interesting is that qualitative findings from the parent
interviews and focus groups indicated that, in general, parents experienced fewer opportunities
to be involved or engaged at their child’s elementary schools as they experienced at Educare.
Furthermore, there was often a stark contrast between the close, open relationships they had
had with Educare Chicago staff (both teaching and family support staff) and the interactions
and communication they experienced with staff at their child’s current school.
Another key finding was that the majority of Educare parents described positive parenting
attitudes, practices, and beliefs. The parents reported a willingness to communicate with teachers
about their child and described specific ways in which they tried to support their child’s learning
in the home. The interview and focus group data indicated that the majority of parents continued
to reflect on their parenting style and how it affected their child in elementary school. For
example, parents reported concerns for their child’s social-emotional development given the
stressors in their families, schools, and communities and provided examples of how they tried
to buffer their child from those stressors to protect the child’s development. Parents described
how they valued education as a ‘‘way to a better future’’ for their child, tried to support their
child’s learning by being available to assist with homework, and recognized the positive impact
34 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
on their child of communicating with their child’s teacher. Parents credited learning these values
and strategies for supporting their child—and, in particular, how to effectively communicate
with educators and school personnel—to their experiences with Educare staff. A large body
of evidence now exists that demonstrates the role of strong home–school connections and parent
responsibility for children’s learning in promoting positive outcomes for children (Harvard
Family Research Project, 2006; Rodriguez & Tamis-LeMonda, 2011), which gives even greater
value to the Follow-Up Study findings and Educare staff efforts with families (described below).
The parent interviews and focus groups also identified common and persistent stressors that
families experienced. Financial and relationship stresses (including loss of extended family or
coparenting support through death or incarceration) were pivotal in that, if relieved, parents
reported a strengthened focus on parenting and supporting their child in school. Isolated parents
who experienced these stressors were less likely to navigate them successfully, less able to be sup-
portive of their child or advocate for their child when needed, and less likely to attend to their par-
enting behavior. These findings provided evidence of the importance of connecting parents to one
another in hopes of forming long-lasting bonds and social networks (Small, 2009). Furthermore,
Reynolds et al. (2010) noted that many children are entering schools at higher risk than students
entering 10 years ago and recommended a decade, starting at birth, of continuous services, includ-
ing a strong parent support component in order to provide the optimal level of support for children’s
learning and development. They further pointed out that intervention at any stage of development
(infancy, preschool, school age) alone cannot prevent children from future underachievement.
Qualitative findings from the Follow-Up Study indicated that parents of children with special
needs often experienced time delays and other barriers that prevented their children from
continuing to receive the needed special education services in elementary school. When these
children were attending Educare, Educare staff assisted parents through the process of qualifying
for and accessing special education services and supports as well as establishing an individua-
lized and appropriate education plan. After their children’s graduation from Educare, many par-
ents with children with identified special needs felt that they continued to need additional
support and advocacy to get needed services, a trend confirmed in the research literature (Healy,
Keesee, & Smith, 1985; Kayama, 2010).
Program response. Educare Chicago staff and the RPP team recognized that these
Follow-Up Study findings gave further credence to the family-centered approach and onsite fam-
ily engagement and support opportunities offered by Educare. Therefore, program activities
centered on continuing to bolster partnerships with families, supports to help families be better
able to navigate multiple life stressors, and opportunities for continued learning. Specifically,
these included the following:
. increased family engagement efforts, including more opportunities for parents to learn
about specific strategies to support and develop the knowledge and skills necessary
for their child to be school ready;
. training in the Brazelton Center’s Touchpoints approach to parent–professional interac-
tions, which orients program staff to view parents as equal partners with knowledge and
skills to be shared with the program (Sparrow, 2011);
. a plan for Educare’s Disabilities Coordinator to become more involved in planning tran-
sition activities, in searching for schools that have adequate special education resources,
USING DATA TO ENHANCE QUALITY IMPROVEMENT 35
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
and in finding advocacy and parent support networks and organizations that can assist
parents in elementary school and beyond; and
. the provision of learning opportunities for current and graduate Educare parents to learn
about the six protective factors that can help build stronger families (Horton, 2003).
With a growing literature base and a clearer understanding of the ‘‘how’’ of family engage-
ment and its associated outcomes (Barnard, 2004; Fantuzzo, McWayne, & Perry, 2004; Harvard
Family Research Project, 2006), an exploration and planning process across the Ounce of Pre-
vention Fund is under way to develop and implement a comprehensive new framework focused
on family, school, and community engagement. The Ounce and Educare Chicago leadership are
also exploring innovative strategies for parents to connect with and learn from one another—to
form social networks that provide parents with a broader set of resources and social capital
(Small, 2009). The finding related to continued, persistent stressors and isolation for families
is a challenging and complex issue with few easy solutions. Continued analysis of this pattern
with further cohorts and larger sample sizes may help isolate other factors that could help ident-
ify families who need more targeted strategies and interventions while at Educare.
DISCUSSION
A Cohesive Educational Experience and Other Challenging Issues
An overarching lesson learned from the cumulative experiences of Educare families as they tran-
sitioned to kindergarten and beyond was the importance of providing students with the highest
levels of need with a cohesive educational experience, beginning at birth and continuing into
elementary school and beyond, mirroring the efforts of preschool to elementary school align-
ment (Bogard & Takanishi, 2005; Marietta, 2010; Reynolds, 2003; Reynolds & Temple,
2008). To help address this need, the Ounce of Prevention Fund and its directly operated
Educare Chicago School, including members of the Educare Chicago RPP team, formed a
new partnership with the University of Chicago’s Urban Education Institute (UEI), which oper-
ates two nearby University of Chicago Charter Schools (UCCS) to align standards, curriculum,
pedagogy, assessments, and family engagement practices from birth to college, with an initial
focus on the first 8 years of life.
In 2010, the Ounce of Prevention Fund, Educare Chicago, UEI, and UCCS staff began this
important new work through a series of professionally facilitated meetings and site visits to build
mutual understanding of shared vision and goals and the trust among staff to pursue them.
UCCS and Educare Chicago also successfully arranged to coordinate admissions policies and
procedures in a way that allowed kindergarten-bound Educare students to transition directly into
one of UCCS’s two elementary charter schools if the parent chose either of these schools for
their child’s elementary education. Almost 90% of Educare families with age-eligible children
chose to participate in this coordinated admissions process and have the option of enrolling in
one of the two UCCS schools in Fall 2014. The Follow-Up Study indicated that prior to the
development of coordinated admissions processes, 37 Educare graduates had enrolled at UCCS
via the charter school lottery system. This partnership will also allow school staff involved in
this birth-to-college partnership to intentionally focus on aligning supports and services for
36 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
children with special needs or children who have been identified as potentially benefitting from
special education services in the primary grades.
Although this will be a multiyear, intensive collaboration, the combined vision is to build a
seamless demonstration model of public education for children and their families that begins at
birth and creates success in school, college, and life. The overarching goal is to align and create
instructional approaches and academic and social supports to accelerate student learning while
honoring and building upon the strengths of the families served. As this model continues to be
developed, we will analyze Follow-Up Study data for the subgroup of children who transition
from Educare to the two UCCS schools to provide data and insights on the impact of this
birth-to-college model.
Despite this promising new model, Educare Chicago staff recognize that there are too many
low-performing schools in local neighborhoods and that some graduates will continue to tran-
sition into schools that will not adequately support the academic, social, and behavioral gains
children have made at Educare. The partnership with University of Chicago’s UEI and UCCS
programs offers just one solution to this stark reality but will not reach all of Educare’s
kindergarten-bound children and families. It is important to note that a primary objective of
the birth-to-college partnership is to document and disseminate both written and video-based
products that clearly describe components of this work, delineate lessons learned, and articulate
implications for best practice and systems and policy change with the ultimate goal of advancing
the replication of this model. Nonetheless, Educare program staff recognize the necessity of
locating additional elementary schools interested in working to create deeper transition align-
ments that will support the successful growth and development of children and their families
from birth through elementary school and beyond.
Educare program staff also realize that a part of the difficulty with transitioning from an early
childhood program to an elementary school setting stems from entering classrooms that have
more children per teacher and subsequently less individualized teacher guidance and interaction
than in Educare classrooms. Dedication to maintaining a high-quality birth-to-5 program and
school cannot necessarily mitigate these two factors. In addition, early education pedagogy
informs them that children learn best in environments that offer choices; incorporate a variety
of learning opportunities; and provide individual, small-group, and large-group experiences
throughout the day (Copple & Bredekamp, 2009). Leadership and teaching staff struggle with
how to keep that learning environment intact and still prepare children for the demands of sitting
longer and attending independently to classroom assignments and activities in kindergarten.
Alumni Parents as a Resource
The sample of graduate parents, or alumni, as they are increasingly called, has become an unin-
tended resource and opportunity for Educare Chicago and the Ounce of Prevention Fund.
Through the study, this sample of families has remained connected to Educare long past their
child’s tenure in the school. Most parents credited Educare with helping their child and them-
selves be better prepared for elementary school and weathering the challenges they encountered.
The parents also retained many fond memories of the staff, events, and other parents while their
child or children attended Educare. During the interviews and focus groups, many parents voiced
a desire to give back, to volunteer in ways that could help current parents and children, from
USING DATA TO ENHANCE QUALITY IMPROVEMENT 37
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
helping to orient new parents to Educare to supporting transitioning parents. Continued contact
with the parents through the interviews, focus groups, and the annual parent event also rein-
forced the Educare message of staying connected to their child’s school and teacher and staying
informed about how the child is experiencing school and achieving in school. Providing support
in elementary school that builds on the foundation established during the birth-to-5 years at Edu-
care can be effective in supporting student gains among low-income children (Layzer, Goodson,
Bernstein, & Price, 2001). Plans to incorporate alumni parents into activities at Educare with
current parents and to capitalize on their desire to offer their expertise and experience to other
parents are still in development. Whatever activities result, they will be another example of
quality improvements to Educare Chicago as result of data from the Follow-Up Study.
CONCLUSION
The requirement for data-based decision making in education and, more recently, in early childhood
education, is ubiquitous (Orszag, 2009; Snow & Van Hemel, 2008). The RPP model practiced at
Educare Chicago was described, and the Educare Chicago Elementary School Follow-Up Study illu-
strated how the partnership worked to improve program practices by using data collected on the
elementary school experiences and progress of children who had graduated from Educare. Explora-
tory and largely descriptive, the Follow-Up Study benefitted Educare Chicago in many ways and
informed specific areas of program improvement, including activities to support parents’ knowledge
about school quality and the application process to apply for enrollment in higher quality options;
classroom activities and explorations for children in the months prior to their transition to
kindergarten; an early math professional development initiative for teachers of infants, toddlers,
and preschoolers; and an understanding of the need to better support children’s approaches to learning
and how to partner more effectively with parents during critical transition points. The Educare pro-
gram leadership views the Follow-Up Study as a constructive catalyst for new thinking and practices.
The Follow-Up Study data also revealed areas that continue to challenge Educare’s program
improvement. Identification of services and specialized supports for families with limited resources
to navigate complex life stressors, continued advocacy support for special needs children, and the
provision of a cohesive educational experience into elementary school and beyond for all Educare
graduates continue to be challenges with no simple solutions. However, the RPP serves as a venue in
which these issues can be discussed and clarified. Resultant strategies and interventions can then be
attempted and analyzed for results that will lead to further refinements or other solutions.
Findings from the Educare Chicago Elementary School Follow-Up Study and the associated
program responses generated through a collaborative, data-driven RPP offer a useful model for
early childhood education practitioners, programs, and policymakers for effectively evaluating
and utilizing data on students and families who have entered elementary school. This can serve
as a model for how to use both K–3 student program data and parent qualitative data to inform
practices in the early years.
ACKNOWLEDGMENTS
The work described in this article has been generously funded by the W. Clement & Jessie V.
Stone Foundation and the Alvin H. Baum Family Fund. We would like to express our deepest
38 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
gratitude for the tremendous efforts and contributions of the Educare Chicago Elementary
School Follow-Up Study research staff (Jacqueline Robinson, Edith Brown, and Todd Jackson)
and the collaboration, thoughtfulness, and hard work of Educare Chicago leadership and staff.
Finally, we would like to thank our research participants, the Educare Chicago graduates and
their families, for their valuable time, insights, and opinions.
REFERENCES
Barnard, W. M. (2004). Parent involvement in elementary school and educational attainment. Children & Youth ServicesReview, 26, 39–62. doi:10.1016/j.childyouth.2003.11.002
Bogard, K., & Takanishi, R. (2005). PK-3: An aligned and coordinated approach to education for children 3 to 8 years
old. Social Policy Report, 19(3), 3–23. Retrieved from the Society for Research in Child Development website: http://
www.srcd.org/documents/publications/SPR/spr19-3.pdf
Coleman, M., Buysse, V., & Neitzel, J. (2006). Recognition and response: An early intervening system for young
children at-risk for learning disabilities. Retrieved from the University of North Carolina at Chapel Hill, Frank Porter
Graham Child Development Institute website: http://www.recognitionandresponse.org/images/downloads/2006
fpgsynthesis_recognitionandresponse.pdf
Copple, C., & Bredekamp, S. (Eds.). (2009). Developmentally appropriate practice in early childhood programs (3rd
ed.) Washington, DC.
Cross, C. T., Woods, T. A., & Schweingruber, H. (Eds.). (2009).Mathematics learning in early childhood: Paths towardexcellence and equity. Washington, DC: National Academies Press.
Downs, A., & Strand, P. S. (2006). Using assessment to improve the effectiveness of early childhood education. Journal
of Child and Family Studies, 15, 671–680. doi:10.1007/s10826-006-9080-7
Duncan, G. J., Dowsett, C. J., Claessens, A., Magnuson, K., Huston, A. C., Klebanov, P., . . . Japel, C. (2007). Schoolreadiness and later achievement. Developmental Psychology, 43, 1428–1446. doi:10.1037/0012-1649.43.6.1428
Dunn, L., & Dunn, D. (2007). Peabody Picture Vocabulary Test. (4th ed.) Minneapolis, MN.
Fantuzzo, J., McWayne, C., & Perry, M. (2004). Multiple dimensions of family involvement and their relations to
behavioral and learning competencies for urban, low-income children. School Psychology Review, 33, 467–480.
Fuchs, D., Mock, D., Morgan, P. L., & Young, C. L. (2003). Responsiveness-to-intervention: Definitions, evidence, and
implications for the learning disabilities construct. Learning Disabilities Research & Practice, 18, 157–171.
doi:10.1111/1540-5826.00072
Fuligni, A. S., Howes, C., Lara-Cinisomo, S., & Karoly, L. (2009). Diverse pathways in early childhood professional
development: An exploration of early educators in public preschools, private preschools, and family child care
homes. Early Education & Development, 20, 507–526. doi:10.1080/10409280902783483
Gresham, F. M. (2007). Evolution of the response-to-intervention concept: Empirical foundations and recent develop-
ments. In S. Jimerson, M. Burns, & A. VanDerHeyden (Eds.), Handbook of response-to-intervention: The science
and practice of assessment and intervention, (pp. 10–24). New York, NY: Springer Science.
Halle, T., Forry, N., Hair, E., Perper, K., Wandner, L., Wessel, J., & Vick, J. (2009). Disparities in early learning and
development: Lessons from the Early Childhood Longitudinal Study–Birth Cohort (ECLS-B). Retrieved from the
Child Trends website: http://www.childtrends.org/Files/Child_Trends-2009_07_10_ES_DisparitiesEL.pdf
Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday experiences of young American children.
Baltimore, MD: Brookes.
Harvard Family Research Project. (2006). Family involvement in early childhood education. Family Involvement Makes
a Difference, 1, 1–8. Retrieved from the Harvard Family Research Project website: http://www.hfrp.org/
publications-resources/browse-our-publications/family-involvement-in-early-childhood-education
Healy, A., Keesee, P. D., & Smith, B. J. (1985). Early services for children with special needs: Transactions for familysupport. Baltimore, MD: Brookes.
Honig, M. I., & Coburn, C. (2008). Evidence-based decision-making in school district central offices: Toward a policy
research agenda. Educational Policy, 22, 578–608. doi:10.1177/0895904807307067
Horton, C. (2003). Protective factors literature review: Early care and education programs and the prevention of childabuse and neglect. Retrieved from the Center for the Study of Social Policy website: http://www.strengtheningfami-
liesillinois.org/mirror/downloads/Literature%20Review_Horton.pdf
USING DATA TO ENHANCE QUALITY IMPROVEMENT 39
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
Horton, C., & Bowman, B. (2002). Child assessment at the preprimary level: Expert opinion and state trends. Retrievedfrom the Erikson Institute website: http://www.erikson.edu/default/research/researchpubs.aspx
Improving Head Start for School Readiness Act of 2007 (P.L. 110–134), 42 U.S.C. x 9801 et seq.
Kagan, S. L. (2003). Children’s readiness for school: Issues in assessment. International Journal of Early Childhood, 35,
114–120. doi:10.1007/BF03174437
Kayama, M. (2010). Parental experiences of children’s disabilities and special education in the U.S. and Japan:
Implications for school social work. Social Work, 55, 117–125. doi:10.1093/sw/55.2.117
Kincaid, D., Childs, K., Blase, K. A., & Wallace, F. (2007). Identifying barriers and facilitators in implementing
school-wide positive behavior support. Journal of Positive Behavior Interventions, 9, 174–184. doi:10.1177/
10983007070090030501
Kraft-Sayre, M. E., & Pianta, R. C. (2000). Enhancing the transition to kindergarten: Linking children, families and
schools. Charlottesville: University of Virginia, National Center for Early Development & Learning.
Layzer, J. I., Goodson, B. D., Bernstein, L., & Price, C. (2001). National evaluation of family support programs: Volume
A. The meta-analysis. Final report. Retrieved from the U.S. Department of Health & Human Services, Administration
for Children & Families, Office of Research, Planning, & Evaluation website: http://www.acf.hhs.gov/programs/opre/
abuse_neglect/fam_sup/reports/famsup/fam_sup_vol_a.pdf
LeBuffe, P. A., Shapiro, V. B., & Naglieri, J. A. (2009). Devereux Student Strengths Assessment: A measure of
social-emotional competencies of children in kindergarten through eighth grade. Lewisville, NC: Kaplan Early
Learning.
Love, J. M., Kisker, E. E., Ross, C., Raikes, H., Constantine, J., Boller, K., . . . Vogel, C. (2005). The effectiveness of
Early Head Start for 3-year old children and their parents: Lessons for policy and programs. Developmental
Psychology, 41, 885–901. doi:10.1037/0012-1649.41.6.885
Marietta, G. (2010). Lessons in early learning: Building an integrated pre-K–12 system in Montgomery County,Maryland. Retrieved from the Foundation for Child Development website: http://www.fcd-us.org/resources/
lessons-early-learningbuilding-integrated-pre-k-12-system-montgomery-county-public-school
National Association for the Education of Young Children, & National Association of Early Childhood Specialists in
State Departments of Education. (2003). Early childhood curriculum, assessment, and program evaluation: Positionstatement with expanded resources. Retrieved from http://www.naeyc.org/files/naeyc/file/positions/CAPEexpand.pdf
National Center for Education Statistics. (2001). America’s kindergartners: Findings from the Early Childhood
Longitudinal Study–Kindergarten Class of 1998–99 (Publication No. 2001-070R). Retrieved from http://nces.
ed.gov/pubs2000/2000070.pdf
Nelson, S. R., Leffler, J. C., & Hansen, B. A. (2009). Toward a research agenda for understanding and improving use of
research evidence. Retrieved from the Education Northwest website: http://educationnorthwest.org/content/about
No Child Left Behind Act of 2001 (P.L. 107–110), 20 U.S.C. x 6301 et seq.Orszag, P. (2009, October 7). Re: Increased emphasis on program evaluation [Memorandum]. Retrieved from the Office
of Management and Budget website: http://www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/
m10-01.pdf
Patton, M. Q. (2008). Utilization-focused evaluation. (4th ed.) San Francisco, CA.
Reynolds, A. J. (2003). The added value of continuing early intervention into the primary grades. In A. J. Wang, & H. J.
Walberg (Eds.), Early childhood programs for a new century (pp. 163–196). Washington, DC: Child Welfare League
of America Press.
Reynolds, A. J., Bezruczko, N., & Hagemann, M. (1997). Chicago longitudinal study of children in the Chicago publicschools: User’s guide (Version 5). Madison, WI: University of Wisconsin–Madison and Chicago Public Schools.
Reynolds, A. J., Magnuson, K., & Ou, S. (2010). Preschool-to-third grade programs and practices: A review of research.
Children and Youth Services Review, 32, 1121–1131. doi:10.1016/j.childyouth.2009.10.017Reynolds, A. J., Ou, S., & Topitzes, J. (2004). Paths of effects of early childhood intervention on educational attainment
and delinquency: A confirmatory analysis of the Chicago Child-Parent Centers. Child Development, 75, 1299–1328.
doi:10.1111/j.1467-8624.2004.00742.x
Reynolds, A. J., & Temple, J. A. (2008). Cost-effective early childhood development programs from preschool to third
grade. Annual Review of Clinical Psychology, 4, 109–139. doi:10.1146/annurev.clinpsy.3.022806.091411
Rodriguez, E. T., & Tamis-LeMonda, C. S. (2011). Trajectories of the home learning environment across the first five
years: Associations with children’s vocabulary and literacy skills at pre-kindergarten. Child Development, 82,
1058–1075. doi:10.1111/j.1467-8624.2011.01614.x
40 STEIN ET AL.
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13
Rous, B., McCormick, K., Gooden, C., & Townley, K. F. (2007). Kentucky’s early childhood continuous assessment and
accountability system: Local decisions and state supports. Topics in Early Childhood Special Education, 27, 19–33.
doi:10.1177/02711214070270010201
Ryan, K. E., & DeStefano, L. (Eds.). (2000). Evaluation as a democratic process: Promoting inclusion, dialogue, and
deliberation (No. 85). San Francisco, CA: Jossey-Bass.
Sandall, S. R., Schwartz, I. S., & LaCroix, B. (2004). Interventionists’ perspectives about data collection in integrated
early childhood classrooms. Journal of Early Intervention, 26, 161–174. doi:10.1177/105381510402600301
Schulting, A. B., Malone, P. S., & Dodge, K. A. (2005). The effect of school-based kindergarten transition policies and
practices on child academic outcomes. Developmental Psychology, 41, 860–871. doi:10.1037/0012-1649.41.6.860
Schwartz, I. S., & Olswang, L. B. (1996). Evaluating child behavior change in natural settings: Exploring alternative
strategies for data collection. Topics in Early Childhood Special Education, 16, 82–101.
doi:10.1177/027112149601600108
Scott-Little, C., Kagan, S. L., & Clifford, R. M. (Eds.). (2003). Assessing the state of state assessments: Perspectives on
assessing young children. Retrieved from the University of North Carolina Greensboro, SERVE CENTER website:
http://www.serve.org/FileLibraryDetails.aspx?id=77
Small, M. L. (2009). Unanticipated gains: Origins of network inequality in everyday life. New York, NY: Oxford Uni-
versity Press.
Snow, C. E., & Van Hemel, S. B. (Eds.). (2008). Early childhood assessment: Why, what, and how? A report of the
National Research Council of the National Academies. Washington, DC: National Academies Press.
Sparrow, J. (2011). Child justice, caregiver empowerment, and community self-determination. In B. S. Fennimore, & A.
L. Goodwin (Eds.), Promoting social justice for young children (pp. 35–46). New York, NY: Springer.
Stedron, J. (2009). A look at Maryland’s early childhood data system. Retrieved from the National Conference of State
Legislatures website: http://www.ncsl.org/documents/educ/MDReport.pdf
Stiggins, R. (1999). Evaluating classroom assessment training in teacher education programs. Educational Measurement:
Issues and Practice, 18(1), 23–27. doi:10.1111/j.1745-3992.1999.tb00004.x
Tseng, V. (2012). The uses of research in policy and practice. Social Policy Report, 26(2), 1–24. Retrieved from the
Society for Research in Child Development website: http://www.srcd.org/index.php?option=com_content&task=vie-
w&id=232&Itemid=658
U.S. Department of Education, & U.S. Department of Health, & Human Services. (2011). Race to the Top–Early Learn-
ing Challenge program: Executive summary. Retrieved from http://www.ed.gov/sites/default/files/rtt-elc-draft-
execsumm-070111.pdf
West, J. L., Malone, L., Hulsey, N., Aikens, L., & Tarullo, L. (2010). ACF-OPRE Report: Head Start children go to
kindergarten. Retrieved from the U.S. Department of Health and Human Services, Administration for Children
and Families, Office of Planning, Research and Evaluation website: http://www.acf.hhs.gov/sites/default/files/opre/
hs_kindergarten.pdf
Woodcock, R. W., McGrew, K. S., & Mather, N. (2001). Woodcock-Johnson III Tests of Achievement. Itasca, IL:
Riverside.
Yazejian, N., & Bryant, D. M. (2012). Educare implementation study findings—August 2012. Retrieved from the
University of North Carolina Chapel Hill, Frank Porter Graham Child Development Institute website: http://
www.fpg.unc.edu/sites/default/files/resources/reports-and-policy-briefs/FPG%20Demonstrating%20Results%20-%20
August%202012%20-%20Final.pdf
USING DATA TO ENHANCE QUALITY IMPROVEMENT 41
Dow
nloa
ded
by [
Uni
vers
ity o
f M
inne
sota
Lib
rari
es, T
win
Citi
es]
at 1
0:01
23
Nov
embe
r 20
13