orientation to measuring child and family outcomes for new people

Early Childhood Outcomes CenterEarly Childhood Outcomes Center

Orientation to Measuring Orientation to Measuring Child and Family Outcomes Child and Family Outcomes

for New Peoplefor New People

Orientation to Measuring Orientation to Measuring Child and Family Outcomes Child and Family Outcomes

for New Peoplefor New People

Kathy Hebbeler, ECO at SRILynne Kahn, ECO at FPG/UNC

Early Childhood Outcomes MeetingBaltimore, Maryland August 2007

Early Childhood Outcomes CenterEarly Childhood Outcomes Center 2

We will coverWe will cover

Why the federal government is interested in outcomes

How these outcomes were selected

Review the SPP/APR indicators and what they mean

Questions encouraged throughout!!


How Looking at Child Indicators How Looking at Child Indicators

for EI/ECSE Became Importantfor EI/ECSE Became Important


Focus on AccountabilityFocus on Accountability

Followed an era of focusing on and measuring actions/activities

Call for interest in the “ultimate result”: child and family outcomes

Cut across both the public and private sectors

Funders want data to determine whether a program is doing what it is supposed to do


Critical Events in EI/ECSE Critical Events in EI/ECSE Accountability Accountability 1992 –Osborne and Graebler, Reinventing

Government 1993 – GPRA (Government Performance and

Results Act) passed Intervening years…. 2002 – PART (Program Assessment Rating

Tool) finds there are no data on outcomes for Part C or Part B Preschool

2003 – OSEP begins to ask states for child outcome data (and funds the ECO Center!)

2005, 2006 – OSEP releases revisions to the reporting requirements


PART Review for Part C and Part PART Review for Part C and Part B PreschoolB Preschool

Results Not Demonstrated

Part C“While the program has met its goal relating to the number of children served, it has not collected information on how well the program is doing to improve the educational and developmental outcomes of infants and toddlers served.”

Part B Preschool“The Department has no performance information on preschool children with disabilities served by this program.”

Read more at Expectmore.gov


What was happening in the What was happening in the intervening years?intervening years? Special Education –

National study found poor outcomes for special education adolescents Results

Push to include students with disabilities in statewide assessment systems

Early Childhood Debate about whether child outcomes

should be measured at all Discussion of the many problems in trying

to measure outcomes for young children with disabilities


Measuring Child Indicators for Measuring Child Indicators for EI/ECSEEI/ECSE

The PART findings put an end to the debate about whether or not to do it

Unfortunately, almost no progress had been made in the intervening years as to HOW to do it


Why These Outcomes?


Early Childhood Outcomes Early Childhood Outcomes CenterCenter

Funded in October 2003 Promote the development and

implementation of child and family outcome measures for infants, toddlers and preschoolers with disabilities that can be used in national and state accountability systems


Origin of the Child OutcomesOrigin of the Child Outcomes

ECO Center convened stakeholder meetings followed by public comment period (2004)

First, collected themes and ideas Then, drafted and re-drafted wording

based on input from stakeholders Made recommendation to OSEP (2005)


Themes from the Stakeholders: Themes from the Stakeholders: Child OutcomesChild Outcomes

Consistent with IDEA and legislative intent

Reflect what EI and ECSE are trying to do

One set for birth to 5 Reflect what is known about

development and learning


Themes from the StakeholdersThemes from the Stakeholders

Be compatible with best practice (esp. transdisciplinary service models, functional behaviors) Do not base them on domains

Have potential to influence practice in a positive way

Incorporate universal design Be readily understood


Make Outcomes/Indicators Make Outcomes/Indicators Functional Functional

Functional refers to things that are meaningful to the child in the context of everyday living

Refers to an integrated series of behaviors or skills that allow the child to achieve the outcomes

Not discrete skills; not domain-based


RecommendationsRecommendations

Decision: One set of indicators birth to 5

Decision: Functional outcomes Decision: Global, not specific


The need for an introductory The need for an introductory statementstatement

Family and child outcomes are linked There are overarching goals for children

and family that cut across the outcomes Helpful to frame the outcomes with

these overarching goals


Concepts in the introduction: Goal for Concepts in the introduction: Goal for childrenchildren

The ultimate goal is for young children to be active and successful participants now and in the future in a variety of settings– in their homes, in their child care, preschool or school programs, and in the community

http://www.fpg.unc.edu/~eco/pdfs/eco_outcomes_4-13-05.pdf


Concepts in the introduction: Concepts in the introduction: Goal for childrenGoal for children

Active and successful participants Now and in the future In variety of settings


Concepts in the introduction: Concepts in the introduction: Goal for familiesGoal for families

Enable families to provide appropriate care for their child

Have resources they need to participate in community activities


Concepts in the preface: Concepts in the preface: Outcomes and accountabilityOutcomes and accountability

Outcomes determined by variety of factors

Not all families and children will achieve all outcomes

BUT, system should still strive to achieve them


ECO Child OutcomesECO Child Outcomes

Children have positive social relationships

Children acquire and use knowledge and skills

Children take appropriate action to meet their needs


ECO Family OutcomesECO Family Outcomes

Understand their child’s strengths, abilities, and special needs

Know their rights and advocate effectively for their children

Help their children develop and learn Have support systems Access desired services, programs,

activities in their community


What kind of data should be What kind of data should be reported on child outcomes?reported on child outcomes?

Had extensive discussions with stakeholders about status vs. progress data Status – where child is Progress – how child has changed

Progress data alone is weak evidence How much progress should be

considered “good progress” or “enough progress?”


ECO Recommendations to OSEPECO Recommendations to OSEP

ECO submitted recommendations to OSEP in May 2005 on the kind of percentages that should be requested from states.

http://www.fpg.unc.edu/~eco/pdfs/ECO_response_to_OSEP_5-9-05.pdf


The APR/SPP Requirements Related to Outcomes


OSEP Reporting Requirements: OSEP Reporting Requirements: Child OutcomesChild Outcomes

Positive social emotional skills (including positive social relationships)

Acquisition and use of knowledge and skills (including early language/ communication [and early literacy])

Use of appropriate behaviors to meet their needs

Part C #3; Part B #4


OSEP Reporting CategoriesOSEP Reporting Categories

Percentage of children who: a. Did not improve functioningb. Improved functioning, but not sufficient to

move nearer to functioning comparable to same-aged peers

c. Improved functioning to a level nearer to same-aged peers but did not reach it

d. Improved functioning to reach a level comparable to same-aged peers

e. Maintained functioning at a level comparable to same-aged peers

3 outcomes x 5 “measures” = 15 numbers


Additional information about Additional information about child indicatorschild indicators

Child must be in program at least 6 months

Data must be collected at (near) entry and (near) exit

Entry data for year July 1, 2005 to June 30, 2006 to be reported in February 2007

Progress data to be reported in February 2008 and therafter


Implications for MeasurementImplications for Measurement

Compares entry to exit data for each child- 2 measurement points are needed for each

child for the data analysis. Individual children’s entry data will need to

be maintained and matched to exit data Information about children’s functioning

in each area needs to be compared to age level expectations


Implications for MeasurementImplications for Measurement(continued)(continued)

Outcomes are broad Functional outcomes summarize each

child’s current functioning across settings and situations

Not asking for domains Best practice for assessing young

children recommends the use of multiple measures

Developmental Trajectory: Hypothetical Language Acquisition Rates for Different Groups of Children

0

10

20

30

40

50

60

70

1 6 11 16 21 26 31 36 41 46 51 56Age in Months

La

ng

au

ge

Sc

ore

Typically developing children

Typically developing children (lower)

Children with delays without intervention

Children with delays after intervention

Illustration of 5 Possible Develomental Trajectories (i.e, the OSEP Reporting Categories)

0

10

20

30

40

50

60

70

1 6 11 16 21 26 31 36 41 46 51 56Age in Months

Sco

re

Maintained functioning comparable to age peers

Achieved functioning comparable to age peers

Moved nearer functioning comparable to age peers

Made progress; no change in trajectory

Did not make progress


Comparing to Same Aged Peers?Comparing to Same Aged Peers?

Deficit model? Not individualized? Progress alone is weak evidence Goal of EI/ECSE: Active and successful

participation now and in the future Kindergarten, school readiness, having

friends, community participation Setting high expectations


Comparison of OSEP requirements to Comparison of OSEP requirements to ECO recommendations: Family ECO recommendations: Family outcomes (Part C only)outcomes (Part C only)

Understand their child’s strengths, abilities, and special needs

Know their rights and advocate effectively for their children

Help their children develop and learn

Have support systems Access desired services,

programs, activities in their community

Percent of families participating in Part C who report that EI services have helped the family

Know their rights Effectively communicate

their children’s needs Help their children

develop and learn

ECO OSEP (#4)


OSEP APR Requirements for OSEP APR Requirements for Families for Part B Preschool (#8)Families for Part B Preschool (#8)

Percent of parents with a child receiving special education services who report that schools facilitated parent involvement as a means of improving services and results for children with disabilities

ECO recommends that states collect additional information about families of children 3-5


Additional information about Additional information about family indicatorsfamily indicators

Timing is not specified (e.g., every May, at exit, etc.)

States have options with regard to how they collect the information Survey, interview Which tool


Family Outcome IssueFamily Outcome Issue

OSEP reporting requirements do not call for data on family outcomes for C or B


Family Outcomes IssuesFamily Outcomes Issues

All states are doing some kind of data collection on family issues OSEP requirements + satisfaction +/or parent

involvement +/or outcomes, etc. Some states are opting to collect

data on family outcomes


Tools for Indicator MeasurementTools for Indicator Measurement

Child Outcomes Summary Form (COSF)

Family Outcomes Survey (FOS)

NCSEAM Survey


Finding out more Finding out more

Additional information about OSEP requirement and state activities

ECO and other resources including training materials for the COSF, Family Outcomes Survey, resources on assessment and typical child development, and materials for parents

www.the-eco-center.orgwww.the-eco-center.org

orientation to measuring child and family outcomes for new people

Documents

poor outcomes

child outcomeseco center

measuring child indicators

implementation of child

child outcome data

program assessment rating

eiecse accountability

family outcomescut