04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

1

Pre-K Pre-K Counts in Pennsylvaniain Pennsylvania for Youngster’s Early for Youngster’s Early School Success:School Success:

Authentic Outcomes for an Innovative Prevention and Promotion InitiativeAuthentic Outcomes for an Innovative Prevention and Promotion Initiative[2005-2009][2005-2009]

STEPHEN J. BAGNATO, Ed.DSTEPHEN J. BAGNATO, Ed.DProfessor of Pediatrics & PsychologyProfessor of Pediatrics & Psychology

University of Pittsburgh/Children’s HospitalUniversity of Pittsburgh/Children’s Hospitalsteve.bagnato@chp.edusteve.bagnato@chp.edu

www.earlychildhoodpartnerships.orgwww.earlychildhoodpartnerships.org

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

2

Funding for the Independent Funding for the Independent Program Evaluation Research of Program Evaluation Research of the SPECS Team (1997-2009) by:the SPECS Team (1997-2009) by:

Senior Program Director, Children, Youth, and Families, Senior Program Director, Children, Youth, and Families, Marge PetruskaMarge Petruska

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

3

What Does 30 Years of National What Does 30 Years of National Research Tell Us about the Research Tell Us about the Developmental Course of Developmental Course of

High-risk Children Who Do Not High-risk Children Who Do Not Participate in Quality Participate in Quality

Preschools?Preschools?

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

42009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

4

04/21/23

Research-based Developmental Declines for High-Research-based Developmental Declines for High-Risk Youngsters Not in PreschoolRisk Youngsters Not in Preschool

(Bryant & Maxwell, 1997; Farran, 2000; Campbell & Ramey, 2002)

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

5

What Is Pre-K Counts in What Is Pre-K Counts in Pennsylvania?Pennsylvania?

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

6

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

7

Distinctive Elements of PKC ProgramsDistinctive Elements of PKC Programs

School-community partnershipsSchool-community partnerships Integration of ECE “system”: Pre-K; Head Start; Integration of ECE “system”: Pre-K; Head Start;

Early Intervention; and Child CareEarly Intervention; and Child Care Collaborative school-community leadershipCollaborative school-community leadership Keystone Stars process & standardsKeystone Stars process & standards Ongoing mentoring to improve program quality, Ongoing mentoring to improve program quality,

teaching, and careteaching, and care Collaborative agreements with human service Collaborative agreements with human service

agenciesagencies PAELS curricular standards and indicatorsPAELS curricular standards and indicators Ongoing program/progress monitoring and Ongoing program/progress monitoring and

evaluationevaluation

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

8

Who Are the Children, Families, Who Are the Children, Families, and Programs in PKC?and Programs in PKC?

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

9

Snapshot of Children, Families, ProgramsSnapshot of Children, Families, Programs

21 PKC school-community partnerships21 PKC school-community partnerships across Pennsylvania across Pennsylvania

10,002 children10,002 children, ages 3-6 years; average age= , ages 3-6 years; average age= 4.3 years4.3 years

Time-in-program= average= 11.7 mo. (4-28 mo.)Time-in-program= average= 11.7 mo. (4-28 mo.) Ethnic representation: Caucasians, African-Ethnic representation: Caucasians, African-

Americans, Hispanics, Asians, Native Hawaiian, Americans, Hispanics, Asians, Native Hawaiian, Alaska Native, American Indian, and Multi-ethnic Alaska Native, American Indian, and Multi-ethnic categoriescategories

1113 teachers in 489 classrooms 1113 teachers in 489 classrooms across PAacross PA

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

10

How Do the SPECS How Do the SPECS Authentic Authentic AssessmentAssessment & Program & Program

Evaluation Research Evaluation Research Methods Work in PKC?Methods Work in PKC?

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

11

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

12

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

13

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

14

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

15

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

16

CHILDREN’SCHILDREN’SEARLY LEARNING EARLY LEARNING

OUTCOMESOUTCOMES

PARTNERSHIPPARTNERSHIPMODELMODEL TIME IN TIME IN

INTERVENTIONINTERVENTION

PROGRAMPROGRAMQUALITYQUALITYMENTORINGMENTORING

TEACHERTEACHERBEHAVIORBEHAVIOR

SPECS for PKC Logic ModelSPECS for PKC Logic Model

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

17

““Core Mandates and Research Questions Core Mandates and Research Questions From Stakeholders for SPECSFrom Stakeholders for SPECS

No exclusion of vulnerable preschoolers No exclusion of vulnerable preschoolers from PKC for research purposes—ethical from PKC for research purposes—ethical designdesign

Is participation in PKC associated with Is participation in PKC associated with significant children’s gains in important significant children’s gains in important functional competencies to improve their functional competencies to improve their early school success? early school success? [Did it work?)[Did it work?)

What programmatic elements of PKC are What programmatic elements of PKC are associated with children’s success? associated with children’s success? [Why [Why did it work?]did it work?]

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

18

What Did the SPECS Research What Did the SPECS Research Show About PKC?Show About PKC?

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

19

FAST FACTS forFAST FACTS forOverall PKC Research OutcomesOverall PKC Research Outcomes

Vulnerable young children beat the odds and succeeded.Vulnerable young children beat the odds and succeeded. Young high-risk children showed accelerated early learning Young high-risk children showed accelerated early learning

progress.progress. Young children with delays and challenging behaviors improved Young children with delays and challenging behaviors improved

equally.equally. Young children learned critical functional competencies for early Young children learned critical functional competencies for early

school success and matched or exceeded local historical and school success and matched or exceeded local historical and state and national norms.state and national norms.

Individualized and inclusive programs helped children to succeed.Individualized and inclusive programs helped children to succeed. Mentored programs improved program quality and teaching which Mentored programs improved program quality and teaching which

promoted child success.promoted child success. Standards for children and professional practices served to focus Standards for children and professional practices served to focus

and guide teaching and expected outcomes for teachers.and guide teaching and expected outcomes for teachers. School-community collaborations and leadership were often School-community collaborations and leadership were often

innovative, effective, and value-addedinnovative, effective, and value-added.

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

20

OUTCOME 1: OUTCOME 1:

High-Risk Preschool High-Risk Preschool Children Beat the Odds and Children Beat the Odds and Succeeded in Pre-K Counts Succeeded in Pre-K Counts

by Gaining Critical Early by Gaining Critical Early Learning Competencies.Learning Competencies.

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

212009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

21

04/21/23

Functional Classifications of Children At Functional Classifications of Children At PKC PKC EntryEntry

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

222009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

22

04/21/23

Functional Classifications of Children at Functional Classifications of Children at PKC PKC ExitExit

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

23

0

5

10

15

20

25

30

35

Entry Exit K-Trans

PKC Data

33

14

2

Comparison of US National Comparison of US National Delay/Disability Incidence Rates to Delay/Disability Incidence Rates to

PKC RatesPKC Rates(Fugiura & Yamaki, 2005)(Fugiura & Yamaki, 2005)

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

242009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

24

04/21/23

Gains of PKC Children Compared to Indicators in Gains of PKC Children Compared to Indicators in National ECE ResearchNational ECE Research

[Mean effect size= .46 or 6.8 standard score units][Mean effect size= .46 or 6.8 standard score units]

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

252009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

25

04/21/23

Reductions in % of At-Risk Children with Reductions in % of At-Risk Children with Social BehaviorSocial Behavior Problems After PKC Problems After PKC

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

26

OUTCOME 2:OUTCOME 2:

Improved Program Quality Improved Program Quality Promoted Children’s Early Promoted Children’s Early

School SuccessSchool Success

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

272009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

27

04/21/23

Comparative Child Progress in Comparative Child Progress in High (3-4)High (3-4) vs vs Low Quality (1-2)Low Quality (1-2) PKC Programs PKC Programs

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

282009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

28

04/21/23

Randomized Study of Improvement in PKC Randomized Study of Improvement in PKC Program’s Quality and Instructional Practices Program’s Quality and Instructional Practices

Over 9 monthsOver 9 months

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

29

OUTCOME 3: OUTCOME 3:

Ongoing Mentoring Ongoing Mentoring Improved Teaching and Improved Teaching and

Program QualityProgram Quality

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

302009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

30

04/21/23

Did Keystone Stars Coaching Improve Did Keystone Stars Coaching Improve Teaching and Program Quality?Teaching and Program Quality?

Variety of coaching Variety of coaching modes was partly modes was partly responsible for responsible for improvements in improvements in quality which quality which promoted children’s promoted children’s language and math language and math competencies at K-competencies at K-transitiontransition

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

312009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

31

04/21/23

Frequency of Effective Coaching Modes in Frequency of Effective Coaching Modes in Keystone StarsKeystone Stars

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

32

OUTCOME 4: OUTCOME 4:

Children in PKC Programs Children in PKC Programs Beat Local and National Beat Local and National

Norms to Achieve Success Norms to Achieve Success at Kindergarten Transitionat Kindergarten Transition

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

33

SPECIFIC PAELS COMPETENCYSPECIFIC PAELS COMPETENCY % ATTAINED % ATTAINED

Demonstrate initiative and curiosityDemonstrate initiative and curiosity 8585

Develop and expand listening and understanding Develop and expand listening and understanding skillsskills

8080

Communicate ideas, experiences and feeling for a Communicate ideas, experiences and feeling for a variety of purposesvariety of purposes

8787

Comprehends information from written and oral Comprehends information from written and oral stories and textsstories and texts

7878

Develop increasing understanding of letter Develop increasing understanding of letter knowledgeknowledge

7676

Learn about numbers, numerical representation, and Learn about numbers, numerical representation, and simple numerical operationssimple numerical operations

7373

Develop self-regulationDevelop self-regulation 8181

Attainment of PAELS Indicators at K TransitionAttainment of PAELS Indicators at K Transition

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

342009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

34

04/21/23

Comparative Early Learning Competencies Comparative Early Learning Competencies of PKC Children with National Norms at of PKC Children with National Norms at

K-TransitionK-Transition

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

352009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

35

04/21/23

Reductions in Special Education Placement Rates Reductions in Special Education Placement Rates for PKC Children Compared to Historical for PKC Children Compared to Historical

School District RatesSchool District Rates

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

36

OUTCOME 5: OUTCOME 5:

Innovative School-Innovative School-Community Partnership Community Partnership

Models Nurtured Child and Models Nurtured Child and Program SuccessProgram Success

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

372009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

37

04/21/23

Comparison of K-Transition Skills of PKC Comparison of K-Transition Skills of PKC Children in Programs with Low vs High Children in Programs with Low vs High

Adherence to OCDEL Partnership ElementsAdherence to OCDEL Partnership Elements

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

38

Simply, Prevention Works!Prevention Works!Inclusion Works!Inclusion Works!Pre-K Counts In Pre-K Counts In

Pennsylvania WORKS!Pennsylvania WORKS!

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

39

What Are the “Lessons Learned” What Are the “Lessons Learned” from PKC for Future Policies, from PKC for Future Policies,

Professional Practices, and Professional Practices, and Research in PA and the US?Research in PA and the US?

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

40

Lessons Learned from PKC ResearchLessons Learned from PKC Research Specific features of PKC seem to make a Specific features of PKC seem to make a

difference.difference. Future research is vital to follow PKC children Future research is vital to follow PKC children

into grades K-5 and prove sustainability.into grades K-5 and prove sustainability. A mentoring model and rigorous documentation A mentoring model and rigorous documentation

are needed to enhance Keystone Stars.are needed to enhance Keystone Stars. SPECS for PKC research can help prospective SPECS for PKC research can help prospective

programs make strategic decisions.programs make strategic decisions. PKC partnerships must embrace and include all PKC partnerships must embrace and include all

types of community ECI partners.types of community ECI partners. Inclusion works and benefits all children.Inclusion works and benefits all children.

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

41

Lessons Learned from PKC ResearchLessons Learned from PKC Research Maximize Early Head Start and Head Start as a Maximize Early Head Start and Head Start as a

key part of the foundation for PKC.key part of the foundation for PKC. Response-to-intervention is a key to effective and Response-to-intervention is a key to effective and

integrated service delivery in PKC.integrated service delivery in PKC. Authentic Assessment is the most effective form Authentic Assessment is the most effective form

of measurement for PKC purposes.of measurement for PKC purposes. The best measurement methods for both children The best measurement methods for both children

and contexts must be re-examined for use in the and contexts must be re-examined for use in the PKC system.PKC system.

Commitment to standards underlies the success Commitment to standards underlies the success of PKC.of PKC.

04/21/232009 Early Childhood Partnership, SPECS Evaluation Team, University of Pittsburgh

42

The MisMeasure of Man The MisMeasure of Man (Stephen J. Gould, 1981)(Stephen J. Gould, 1981)

““There are…fewer injustices deeper There are…fewer injustices deeper than the denial of an opportunity to than the denial of an opportunity to

strive or ever hope by a limit strive or ever hope by a limit imposed from without, but falsely imposed from without, but falsely

identified as lying within” identified as lying within” (p.28)(p.28)