Improving Quality Practices in Improving Quality Practices in Early Intervention:Early Intervention:

Missouri’s IFSP Quality Rating Missouri’s IFSP Quality Rating ScaleScale

AgendaAgenda

History / Tie to State Improvement History / Tie to State Improvement PlanPlan

Description of IFSP Quality Indicators Description of IFSP Quality Indicators Rating Scale (QIRS)Rating Scale (QIRS)– Development & implementationDevelopment & implementation

Selecting & Reviewing IFSPsSelecting & Reviewing IFSPs

Preliminary Results of ReviewPreliminary Results of Review

Plans for the futurePlans for the future

SICC-Identified Needs SICC-Identified Needs in First Stepsin First Steps

SICC meetings from 3/03 – 1/04 SICC meetings from 3/03 – 1/04 identified a number of priorities for identified a number of priorities for improvement:improvement:– A system of accountability/oversight to A system of accountability/oversight to

ensure high quality, family centered ensure high quality, family centered servicesservices

– Provider availability and trainingProvider availability and training– Financial/budgetary concerns (cost-Financial/budgetary concerns (cost-

containment)containment)

Overall concerns about the quality of Overall concerns about the quality of services & appropriateness of IFSPsservices & appropriateness of IFSPs

Legislative/State ConcernsLegislative/State Concerns

Program costs increasingProgram costs increasing

Variation in cost per childVariation in cost per child

Limited support/guidance to ensure Limited support/guidance to ensure quality servicesquality services

Need cost containment without Need cost containment without compromising qualitycompromising quality

OSEP InfluencesOSEP Influences

Missouri needs a system of Missouri needs a system of accountabilityaccountability

Service coordination concernsService coordination concerns

Monitoring SPOEs to ensure qualityMonitoring SPOEs to ensure quality

Missouri’s Improvement PlanningMissouri’s Improvement Planning

Stakeholder meeting with NECTAC Stakeholder meeting with NECTAC – June 28-29, 2004June 28-29, 2004

Understand challenges related to increasing service Understand challenges related to increasing service costs, appropriate provision of services, costs, appropriate provision of services, implementation of accountability strategies & implementation of accountability strategies & incentives to change provider practicesincentives to change provider practices

Develop an improvement plan to address challengesDevelop an improvement plan to address challenges

– June 30 – July 1, 2004June 30 – July 1, 2004Develop a “Standard of Practice in Early Intervention” Develop a “Standard of Practice in Early Intervention” tooltool

If we know what “bad” IFSPs look like, then what do If we know what “bad” IFSPs look like, then what do “good” IFSPs look like?“good” IFSPs look like?

IFSP Quality Indicators Rating IFSP Quality Indicators Rating Scale (QIRS)Scale (QIRS)

Designed to be used in accountability Designed to be used in accountability & monitoring procedures& monitoring proceduresSpecific contractual uses for 3 of 24 Specific contractual uses for 3 of 24 System Points of Entry (SPOEs)System Points of Entry (SPOEs)Reviewers will rate randomly Reviewers will rate randomly selected IFSPs on a scale of 1-5selected IFSPs on a scale of 1-5– 1: needs improvement1: needs improvement– 3: acceptable3: acceptable– 5: recommended practice5: recommended practice

IFSP QIRS DevelopmentIFSP QIRS DevelopmentIFSP document:IFSP document:– Workgroup took sectionsWorkgroup took sections

Identified required componentsIdentified required components

Identified poor & quality indicatorsIdentified poor & quality indicators

– Reported initial recommendations to entire Reported initial recommendations to entire stakeholder group; gather consensusstakeholder group; gather consensus

– NECTAC left with butcher block paper & a NECTAC left with butcher block paper & a collection of concepts and ideascollection of concepts and ideas

– NECTAC used stakeholder input to draft QIRSNECTAC used stakeholder input to draft QIRS– Missouri stakeholders reviewed and revisedMissouri stakeholders reviewed and revised

IFSP QIRS DevelopmentIFSP QIRS Development8/19/04: National expert reviewed rating 8/19/04: National expert reviewed rating scalescale

8/23/04: Conference Call with MO 8/23/04: Conference Call with MO stakeholders, NECTAC & national expertstakeholders, NECTAC & national expert

8/31/04: IFSP Rating Scale finalized8/31/04: IFSP Rating Scale finalized

9/22-23/04: NECTAC provided awareness 9/22-23/04: NECTAC provided awareness level training on EI research and the QIRS level training on EI research and the QIRS in St. Louis (videotaped for later statewide in St. Louis (videotaped for later statewide use)use)

Developing GuidanceDeveloping Guidance

October 13-14, 2004: Develop October 13-14, 2004: Develop exemplar for IFSP QIRSexemplar for IFSP QIRS– Used IFSP for a child from VirginiaUsed IFSP for a child from Virginia– NECTAC provided introduction to NECTAC provided introduction to

developing exemplars; facilitated small developing exemplars; facilitated small group work in this areagroup work in this area

November 3, 2004: Internal review November 3, 2004: Internal review of exemplars complete; published to of exemplars complete; published to workgroup for reviewworkgroup for review

Developing GuidanceDeveloping Guidance

Exemplar feedbackExemplar feedback– very positive about having exemplarsvery positive about having exemplars– reworked some in-house revisionsreworked some in-house revisions– clearly requested additional exemplarsclearly requested additional exemplars

Began training SPOEs using scale & Began training SPOEs using scale & exemplar in January 2005exemplar in January 2005

Developing QIRS Review ProcessDeveloping QIRS Review Process

Determining Procedures Determining Procedures – May 3 & 4, 2005May 3 & 4, 2005

NECTAC facilitated stakeholder group in determining NECTAC facilitated stakeholder group in determining process for selecting, reviewing and scoring IFSPs, process for selecting, reviewing and scoring IFSPs, and providing overall scores and feedback to SPOEsand providing overall scores and feedback to SPOEs

Requested files from 3 SPOEs to pilot scoringRequested files from 3 SPOEs to pilot scoring

– May 23, 2005May 23, 2005Piloted scoring – scoring group + Service Piloted scoring – scoring group + Service Coordinators from 2 SPOEs & DMHCoordinators from 2 SPOEs & DMH

– May 27, 2005May 27, 2005Conference call with 3 SPOE Directors regarding Conference call with 3 SPOE Directors regarding proceduresprocedures

Selecting IFSPs for ReviewSelecting IFSPs for ReviewIFSPs are selected up to 10% of SPOE child IFSPs are selected up to 10% of SPOE child count (but no more than 50 files) to count (but no more than 50 files) to sample across:sample across:– Service Coordinators (DMH & SPOE), Service Coordinators (DMH & SPOE), – Initial IFSPs (minimum 5, plan for 1/3 of child Initial IFSPs (minimum 5, plan for 1/3 of child

count), count), – IFSPs that have gone through review (minimum IFSPs that have gone through review (minimum

5, plan for 2/3), 5, plan for 2/3), – IFSPs where the child is 2.5 years or greater IFSPs where the child is 2.5 years or greater

(minimum 5), and (minimum 5), and – IFSPs where AT has been identified as IFSPs where AT has been identified as

necessary (minimum 5).necessary (minimum 5).

IFSP ScoringIFSP ScoringIndividual IFSPs are rated on up to 17 Individual IFSPs are rated on up to 17 areas. areas. – Two areas (Assistive Technology and Two areas (Assistive Technology and

Review) could be not applicable on a Review) could be not applicable on a given IFSP.given IFSP.

– On the 5 point Likert scale, “2” and “4” On the 5 point Likert scale, “2” and “4” are not defined because they represent are not defined because they represent mid-points. mid-points.

– Each IFSP will get a score of 1, 2, 3, 4, or Each IFSP will get a score of 1, 2, 3, 4, or 5 5

IFSP Scoring TableIFSP Scoring Table

Score Score RatingRating All Items All Items (max 73 (max 73 pts)pts)

No No Review Review (max 68 (max 68 pts)pts)

No AT No AT (max (max 65 65 pts)pts)

No No Review or Review or AT (60 AT (60 pts max)pts max)

55 High QualityHigh Quality 68+68+ 64+64+ 61+61+ 56+56+

44 QualityQuality 62-6762-67 58-6358-63 55-6055-60 51-5551-55

33 AcceptableAcceptable 51-6151-61 48-5748-57 45-5445-54 42-5042-50

22 Needs Needs ImprovementImprovement

34-4934-49 32-4732-47 30-4430-44 28-4128-41

11 PoorPoor 17-3317-33 16-3116-31 15-2915-29 14-2714-27

SPOE ScoringSPOE ScoringEach SPOE is rated on overall percentage of IFSPs Each SPOE is rated on overall percentage of IFSPs scored as high quality, quality, acceptable, needs scored as high quality, quality, acceptable, needs improvement, poor.improvement, poor.SPOEs receive a Summary Report listing the SPOEs receive a Summary Report listing the number of files receiving each score. number of files receiving each score. IFSP scores are averaged and the SPOE receives a IFSP scores are averaged and the SPOE receives a rating based on their average IFSP score:rating based on their average IFSP score:– High Quality = 4.5 – 5.0High Quality = 4.5 – 5.0– Quality = 4.0 – 4.4Quality = 4.0 – 4.4– Acceptable = 3.0 – 3.9Acceptable = 3.0 – 3.9– Needs Improvement = 2.0 – 2.9Needs Improvement = 2.0 – 2.9– Poor = 1.0 – 1.9Poor = 1.0 – 1.9

Reviewing IFSPsReviewing IFSPsEach IFSP was evaluated independently by Each IFSP was evaluated independently by 2 reviewers2 reviewersReviewers discussed use of the Rating Reviewers discussed use of the Rating Scale to ensure consistencyScale to ensure consistencyIf scores resulted in the same quality If scores resulted in the same quality range (quality, acceptable, poor, etc.), file range (quality, acceptable, poor, etc.), file completecompleteIf scores in different quality ranges (quality If scores in different quality ranges (quality & acceptable; poor & needs & acceptable; poor & needs improvement), reviewers met and came to improvement), reviewers met and came to consensus on scoreconsensus on score

Preliminary ResultsPreliminary Results1 of 3 SPOEs met “Acceptable” or higher1 of 3 SPOEs met “Acceptable” or higher

IFSPs varied from high quality to poorIFSPs varied from high quality to poor

IFSPs varied within the documentsIFSPs varied within the documents– Service Coordinators seem to have areas of Service Coordinators seem to have areas of

expertiseexpertise– General documentation / completion General documentation / completion

Use of “stock phrases” that don’t change (transition)Use of “stock phrases” that don’t change (transition)

Writing parent comments verbatim/no probingWriting parent comments verbatim/no probing

Lack of tie between family’s concerns, priorities & Lack of tie between family’s concerns, priorities & resources and servicesresources and services

Preliminary ResultsPreliminary Results3 SPOEs average cost per child:3 SPOEs average cost per child:

SPOESPOE July 05July 05 Nov 05Nov 05 ChangeChange

Greater Greater StLStL

$446.96$446.96 $363.74$363.74 (83.22)(83.22)

StL StL CountyCounty

$534.13$534.13 $407.78$407.78 (126.35)(126.35)

NWNW $368.64$368.64 $320.53$320.53 (48.11)(48.11)

Future PlansFuture Plans

Lead Agency and First Steps Lead Agency and First Steps Consultants will meet:Consultants will meet:– Dec 20, 2005: Train LA & FS Dec 20, 2005: Train LA & FS

Consultants on QIRS review processConsultants on QIRS review process– TBD: Review QIRS results to determine TBD: Review QIRS results to determine

statewide and regional training needsstatewide and regional training needs– TBD: Develop & provide trainingTBD: Develop & provide training– Jan/Feb 2006: Embed into service Jan/Feb 2006: Embed into service

coordinator training for 8 new SPOEscoordinator training for 8 new SPOEs

Future PlansFuture PlansLead Agency and First Steps Lead Agency and First Steps Consultants will meet to:Consultants will meet to:– TBD: With TA from NECTAC, evaluate TBD: With TA from NECTAC, evaluate

and revise (if needed) QIRS tool and and revise (if needed) QIRS tool and processprocess

– TBD: Training additional reviewers for TBD: Training additional reviewers for future statewide QIRS implementationfuture statewide QIRS implementation

– TBD: Institutionalize the QIRS reviews as TBD: Institutionalize the QIRS reviews as part of the monitoring & accountability part of the monitoring & accountability processprocess