understanding and using early childhood outcome (eco) data for program improvement tasn – kits...

Understanding and Using Early Childhood Outcome (ECO) Data for Program

Improvement TASN – KITS Fall 2012 Webinar

August 31st, 2012

Tiffany Smith

Phoebe Rinkel

Chelie Nelson

1

2

Tiffany SmithKSDE, ECSE Program [email protected]

Phoebe RinkelKITS, Part B, TA [email protected]

Chelie NelsonKITS, Part B, TA [email protected]

mailto:[email protected]



Online Resourceswww.kskits.org

3

AgendaOverview of the Early Childhood Outcome (ECO)

Data

Kansas Data Drill Down Guide Case StudyExamining Policies and ProceduresExamining APR ReportsExamining ECO Addendum ReportsExamining Data VerificationExamining Child Level Data in OWS

4

Early Childhood Outcomes

OSEP required states to submit outcome data in their State Performance Plan (SPP) and Annual Performance Report (APR)

2010 – 2011 (Federal Fiscal Year 2009) first year Districts and Part C Networks were compared to State targets

5

The Three Early Childhood Outcomes

1. Positive social-emotional skills (including social relationships)

2. Acquisition and use of knowledge and skills (including early language/communication [and early literacy*])

3. Use of appropriate behaviors to meet their needs

*for 3-5 6

How Kansas Early Childhood Outcome Data

is Reported

7

States Report Data in these categories

Percentage of children who:a) Did not improve functioning

b) Improved functioning, but not sufficient to move nearer to functioning comparable to same-age peers

c) Improved functioning to a level nearer to same-age peers, but did not reach it

d) Improved functioning to reach a level comparable to same-age peers

e) Maintained functioning at a level comparable to same-age peers.

8

Entry

Kasprzak & Rooney (2010)

9

Entry Exit


10

Entry Exit


11







12

Entry Exit

a


13

Entry Exit

a


14







15

Entry Exit

b


16

17

Entry Exit

b


Entry Exit

b


18







19

Entry Exit

c


20

Entry Exit

c


21







22

Entry Exit

d

Kasprzak & Rooney (2010)23







24

Entry Exit

e


25

Entry Exit

e


Summary Statements For Reporting Progress on Targets

Required Summary Statement 1:

Of those children who entered or exited the program below age expectations in each Outcome, the percent who substantially increased their rate of growth by the time they exited the program.

c+d __ a+b+c+d

Required Summary Statement 2:

The percent of children who were functioning within age expectations in each Outcome by the time they exited the program.

d+e __ a+b+c+d+e

28

State ECO Targets FY 2010 (Reported on March 2012)

Outcome 1 Outcome 2 Outcome 3

Summary Statement 1

% of children who moved

closer to same age peers

Part C = 57.53%Part B = 86.43%

Part C = 61.14%Part B = 86.88%

Part C = 66.99%Part B = 86.74%

Summary Statement 2

% of children who exited at

age level

Part C = 56.33%Part B = 65.66%

Part C = 47.44%Part B = 64.10%

Part C = 63.44%Part B = 77.29%

29State targets change each year, always be sure to use the most current data for your data drill down

PurposeDeveloped as a tool for local Part B Preschool

Special Education Programs To identify components of a high quality system To evaluate their existing Indicator 7 Data To encourage decision making that will support

program improvement efforts

30

5 Sections A. Local Policies and Procedures for Data Reporting

B. District APR Data

C. Addendum Report Data

D. Data Verification

E. Child Level Data from OWS

31

Each Section includes; Information about the data to be examined and

where it can be foundQuestions to Guide your Review ProcessAction Planning Form

32

Action Plan

33

Suggested Use

Local Implementation TeamPart of an ongoing strategic

planning processMay be completed in total or in

sectionsReassess periodically

34

Many Steps for Ensuring Quality Data

Good Data Collection/Training

Good data system and data entry

Ongoing supervision of implementation

Feedback to implementers

Refresher training

Review of COSF Records

Data Analyses for validity checks

35Kasprzak & Rooney (2010)

Section A: Examine Local Policies and Procedures for

Data Reporting


Data Reporting

Administrator Quality Rating Checklist

Data Entry Quality Rating Checklist

Direct Service Provider Quality Rating Checklist

Questions to Guide the Review Process (pg. 4 Data Drill Down Guide)

36

http://kskits.org/ta/ECOOutcomes/AdminNdsKno/AdminQtyRatingChecklist1_31_12.doc



http://kskits.org/ta/ECOOutcomes/DataEntry/DataEntryQtyRtgCklst1_31_12.doc

http://kskits.org/ta/ECOOutcomes/whatDirectService/DrctSrvcPvdrQtyRtgLst_1_31_12.doc

ECO City Example – HO#6

37


Data Reporting

Section B: Locating and Examining District APR Data

Section B: ECO City APR Data – HO#7

40

Comparing ECO City Data with State Data

7A1 7A2 7B1 7B2 7C1 7C20.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

DistrictState

Section C: Locating and Examining Addendum Reports

41

Section C: Examining Addendum Reports for ECO

City – HO#8

42

Section C: Examining ECO City Addendum Reports –

HO#9

43

Progress and Slippage Reports

Section C: Locating and Examining Addendum

Reports

44

Section D: Data Verification

Data Verification occurs each August 1st – 31st

45

Section D: ECO City Data Verification HO#10

46

Section E: Examining Child Level Data in OWS

47

Section E: Examining Child Level Data in OWS

48

Section E: Parameterized Data Report

49

Section E: No Permanent Exit

50

Section E: Permanent Exit Report

51

Section E: Summary Statement Report

52

53

Section E: ECO Report

54

Sharing Your ECO DataSample messages that can be gleaned from your

ECO data:Data show that children are making progress from entry

to exit in the programMany children are catching up or getting closer to same

age peersPoint out how programs are contributing to school

readinessLink message to broader EC issues (i.e. cost

effectiveness of high quality EC programs)

What the data look like:Nationally

55

Part C and PreschoolAverage Percentage of Children in Each

CategoryOutcome 1: Social/Emotional



CategoryOutcome 2: Knowledge/Skills

Kasprzak & Rooney (2010) 57

58


CategoryOutcome 3: Getting Needs Met


Keeping our eye on the prize: High qualityservices for childrenand families that willlead to good outcomes.


Questions?

62

ReferencesHebbeler, K., Kahn, L., Taylor, C. & Bailey, A.

(2011). Data Workshop: Analyzing and Interpreting Data. Presented at the Measuring and Improving Child and Family Outcomes Conference, New Orleans, LA.

Kasprzak & Rooney (2010, March). Measuring Child Outcomes, Presentation for Delaware; ECO Center & NECTAC. Retrieved 10/3/11 from: http://www.fpg.unc.edu/~eco/assets/ppt/DE%20COSF%20training%20slides%20for%20web%204-12-10.ppt

63

http://www.fpg.unc.edu/~eco/assets/ppt/DE%20COSF%20training%20slides%20for%20web%204-12-10.ppt



understanding and using early childhood outcome (eco) data for program improvement tasn – kits...

Documents

level comparable

age peers e

d improved functioning

entryexit b kasprzak

child level data

entryexit kasprzak rooney

states report data

data verification