1

Assuring the Quality of Data from the Child

Outcomes Summary Form

2

Four-state Panel

• Wendy Whipple (NV) Part C

• Carolee Eslinger and Carrie Mori (ID) Part C

• Ted Maloney and Erica Swanson (MT) Part C

• Margy Hornback, Dale Walker, and Charles Greenwood (KS) Birth-5

3

Each state will describe:

• Approach to COSF quality assurance (QA)

• COSF QA process

• What happens as a result of QA activities

• Issues, challenges, strategies, and lessons learned

4

Nevada’s approach to COSF quality assurance

• Have a task force that meets regularly with representation from all regional programs

• Using a consensus model for decision making

• Have created a matrix for all decisions that have been made to date and a plan for how to implement

5

Nevada’s COSF QA process

• Comparing COSF data with TRAC 618 data to ensure all entries and exits are completed

• Creating reports for regional programs to utilize to manage COSF requirements

• Have instituted statewide training

• Just beginning to look at individual COSF forms to identify concerns

• Revised COSF form to provide assistance to staff in rating

6

What happens in Nevada as a result of your QA activities?

• Creating reports for regional programs

• Analyzing data for children who do not remain in early intervention for six months to see if there are trends

• Just beginning to look at exit COSF forms in comparison with entry

• Determining if the information on the entry form was sufficient to assist with completing an accurate exit form

• Next step to create a quality assurance technical assistance process to address concerns with completion of COSF forms based on review

7

Idaho’s approach to COSF quality assurance

• Developed training and protocol with key stakeholders

• Provided statewide training around the state

Worked closely with Part B to facilitate COSF data sharing

8

Idaho’s COSF QA process

• Generate reports form DataTot to assure entry and exit scores are recorded

• Share reports with Regional Programs

• Planning to conduct a review of data across regions to check for significant discrepancies/scoring errors

9

Idaho’s COSF QA process

Collaborated with Boise State University Early Childhood Studies Program to assess the reliability of the process Scenario created with a brief history, test

scores, comments from parents, and other supporting materials

Regional teams reviewed the scenario, scored the COSF, and provided feedback on the process

Results reported to Central Office for follow-up

10

What happens in Idaho as a result of your QA activities?

Small workgroup convened to recommend process and policy revisions

Proposed revisions shared with SDE (Part B) to assure continued compatibility between systems

Revisions adopted and implemented

Data will continue to be monitored for completeness and quality

11

Montana’s approach to COSF quality assurance

• Review each COSF, team COSF procedures, and agency COSFs for:– Item completion– Composition of COSF teams– Parent/family COSF outcome input– Number of sources for making each COSF

outcome rating– Assessments used for each COSF outcome

rating– Observations/reports used for each COSF rating

12

Montana’s COSF QA process

• COSFs compiled by Part C agency and sent to lead agency on a quarterly basis

• Review every COSF for an agency for each item on the COSF and compile data

• Complete a quarterly report for each agency

• Provide feedback and technical assistance for each agency for each quarter

13

What happens in Montana as a result of your QA activities?

• Provide feedback to each Part C agency, especially concerning systemic issues and positive support

• Provide general feedback to stakeholders• Review for state systems issues• Look for possible future COSF issues (e.g.,

exit ratings)

14

Kansas’ approach to COSF quality assurance

• Developed web-based application for data entry

• Provided training and technical assistance, including documentation of supporting evidence guided by decision tree

• Partnered with researchers for data analysis

15

Kansas’ COSF QA process

• Content analysis of individual child COSF data

• Analysis of data collected in web-based application

• Analysis of video tapes• Survey• Validity studies

16

What happens in Kansas as a result of your QA activities?

Used to improve the quality of the process (technically correct and level of quality)– Refine the web-based application fields– Improve training and technical assistance– Refine research questions– Begin to plan for intervention strategies

to improve outcomes