Considerations for Establishing Baseline and Setting Targets for

Indicators C3 and B7

Kathy Hebbeler,Lynne Kahn, Christina Kasprzak

ECO/NECTACJune 16, 2009

Two related calls

• Last Tuesday, June 9, 2009 on “Review of Summary Statements”– Powerpoints on the ECO website

• Today, “Considerations for Setting Targets”

2

What we’ll cover today

• Two strategies for examining data – Data quality– Potential for program improvement

• Parameters, guidance for target settings from OSEP

3

Can you trust the data?

• Begin by identifying outliers

• Examples: look at the percentages reported for category a and category e across local programs

4

Percentages reported in category “a” across 30 local programs

5

Remove the outliers

• State percentage for “a” with all data= 3.9%

• Revised percentage for “a” with outliers removed= 2.4%

6

Percentages reported in category “e” across 30 local programs

7

Remove the outliers

• State percentage for “e” with all data= 32.1%

• Revised percentage for “e” with outliers removed= 27.7%

8

Example of data with outliers removed

Progress Category

Original % Clean %

a 4 2b 15 17c 27 30d 30 31e 24 20

Sum St 1 75 76Sum St 2 54 51

Clean data (without the outliers) may be a more accurate picture of where you are starting

9

Suggested strategy

• Analyze your data with your local LEA/program outliers included and excluded so you can gauge the impact they are having on your state level data.

Note Note Note

• Consider clean data when deciding about reasonable targets, BUT

• Turn in the original data to OSEP in the SPP report!

• You can discuss the clean data in the rationale for your targets.

11

Which local programs can be targeted for program improvement?

• Compare the summary statement data by local program to identify which programs have the most potential for improvement.

12

Summary Statement Percentages by Local Program

13

Considerations

• What do you know about the programs/LEAs with the least and the most progress in the summary statements? i.e. the programs w/– the lowest and highest percentages of children

at age expectation at exit– the lowest and highest percentage of children

making greater than expected gains14

Examples of Key Questions

• Are the children similar at entry?

• Are the higher performing programs/LEAs participating in special projects? e.g. a state initiative, TACSEI or CELL?

• Are there systems issues in lower performing programs/LEAs that would explain differences in outcomes? e.g. personnel shortages 15

Bottom-line Question

• Could either system or practice focused improvement activities targeted toward the lowest performing programs/LEAs improve the child outcomes?

16

The Math of Target Setting

• How much would the data change if the lowest local programs moved toward the mean?

• Improvements in the lowest programs will result in improvement in your statewide data

• Experiment with your data to determine what targets are reasonable in your state 17

Timelines

• In Feb, 2010, in SPP format: – Baseline– Targets for 2 reporting years– Improvement activities for 2 reporting years

• In Feb, 2011 and 2012, in APR format– Actual data, progress and slippage, etc.– Local reporting of [summary statement %s]

18

Questions and comments?

19

Outcomes Conference

June 22 and 23, Bethesda, MD

Resources at

The-ECO-Center.org

20