assessing fidelity: a necessary component of effective implementation
DESCRIPTION
Assessing Fidelity: A Necessary Component of Effective Implementation. Sandra F. Naoom, MSPH National Implementation Research Network Frank Porter Graham Child Development Institute University of North Carolina- Chapel Hill. Child Care Policy Research Consortium Annual Meeting 2009. - PowerPoint PPT PresentationTRANSCRIPT
Sandra F. Naoom, MSPHNational Implementation Research Network
Frank Porter Graham Child Development Institute
University of North Carolina- Chapel Hill
Assessing Fidelity: A Assessing Fidelity: A Necessary Component of Necessary Component of Effective Implementation Effective Implementation
Child Care Policy
Research Consortium
Annual Meeting 2009
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
What is Implementation?What is Implementation?
The act of The act of accomplishing some aimaccomplishing some aim or or executing some orderexecuting some order
To To put into practical effectput into practical effect; carry out; carry out
Pursue to a conclusionPursue to a conclusion– – Dictionary.comDictionary.com
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Stages of ImplementationStages of Implementation
Major Implementation Initiatives occur in stages:
Exploration
Installation
Initial Implementation
Full Implementation
Innovation
Sustainability
2 – 4 Years
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
INTEGRATED & COMPENSATORY
CONSULTATION & COACHING
CONSULTATION & COACHING
STAFF EVALUATION
STAFF EVALUATION
FACILITATIVE ADMINISTRATIVE
SUPPORTS
FACILITATIVE ADMINISTRATIVE
SUPPORTS
RECRUITMENT AND SELECTION
RECRUITMENT AND SELECTION
PRESERVICE TRAINING
PRESERVICE TRAINING
SYSTEMSINTERVENTIONS
SYSTEMSINTERVENTIONS
Implementation Drivers Implementation Drivers (Processes)(Processes)
DECISION SUPPORT DATA SYSTEMS
DECISION SUPPORT DATA SYSTEMS
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Implementation and FidelityImplementation and Fidelity
“The bridge between a promising idea The bridge between a promising idea and the impact on students is and the impact on students is implementation, but innovations are implementation, but innovations are seldom implemented as intended” .seldom implemented as intended” .
-Berman & McLaughlin, 1976-Berman & McLaughlin, 1976
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Fidelity and Fidelity MeasuresFidelity and Fidelity Measures
Fidelity is the degree to which a Fidelity is the degree to which a program as implemented corresponds program as implemented corresponds with the program as described.with the program as described.
(Fixsen, Naoom, Blase, Friedman & Wallace, 2005, (Fixsen, Naoom, Blase, Friedman & Wallace, 2005,
National Implementation Research Network)National Implementation Research Network)
Fidelity measures are tools to assess Fidelity measures are tools to assess the adequacy of the implementation of the adequacy of the implementation of the program as described. the program as described.
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Fidelity in the Field: The PrevalenceFidelity in the Field: The Prevalence
Implementation efforts in the ‘real’ world, are often Implementation efforts in the ‘real’ world, are often unsuccessful and highly variable both in terms of unsuccessful and highly variable both in terms of the fidelity and sustainability of the program or the fidelity and sustainability of the program or practice. (Elliott & Mihalic, 2004). practice. (Elliott & Mihalic, 2004).
About 23%-81% of program activities may be About 23%-81% of program activities may be omitted during implementation (Durlak, 1998).omitted during implementation (Durlak, 1998).
““Surprisingly, many of the highest-quality programs Surprisingly, many of the highest-quality programs fail to take adequate steps to monitor and verify fail to take adequate steps to monitor and verify program integrity...This weakens the conclusions program integrity...This weakens the conclusions that can be drawn regarding the program outcomes that can be drawn regarding the program outcomes and reduces the likelihood that replications will and reduces the likelihood that replications will resemble the original program” (Domitrovitch and resemble the original program” (Domitrovitch and Greenberg, 2000).Greenberg, 2000).
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Why is Fidelity Important?Why is Fidelity Important?
Basis for program effectivenessBasis for program effectiveness
Helps to avoid Type III ErrorHelps to avoid Type III Error
Promotes Internal and External ValidityPromotes Internal and External Validity
Research has linked fidelity with positive Research has linked fidelity with positive outcomesoutcomes
Adult Mental Health- ACT; Children’s Adult Mental Health- ACT; Children’s Services- FFT, MST, Wraparound; Medicine- Services- FFT, MST, Wraparound; Medicine- DOTS, Texas AlgorithmDOTS, Texas Algorithm
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Why Measure Fidelity?Why Measure Fidelity?
Measure Fidelity AND Outcomes BECAUSEMeasure Fidelity AND Outcomes BECAUSE
you need to know:you need to know:
Are we having an implementation problem?Are we having an implementation problem?
Low fidelity + Poor outcome =Low fidelity + Poor outcome =
Implementation problemImplementation problem
Are we having an effectiveness problem?Are we having an effectiveness problem?
High fidelity + Poor outcome = High fidelity + Poor outcome =
Effectiveness problemEffectiveness problem
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Common Dimensions of Intervention Common Dimensions of Intervention Fidelity Fidelity
AdherenceAdherence
Dosage/ExposureDosage/Exposure
Quality of Program DeliveryQuality of Program Delivery
Participant Responsiveness Participant Responsiveness
Program Differentiation Program Differentiation
(e.g. Dane & Schneider, 1998; Dusenbury, (e.g. Dane & Schneider, 1998; Dusenbury, 2003)2003)
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
NIRN’s Fidelity DimensionsNIRN’s Fidelity Dimensions
Context Context
Pre-requisites and pre-conditions for a Pre-requisites and pre-conditions for a program to operate.program to operate.
ComplianceCompliance
Extent to which practitioner uses intervention Extent to which practitioner uses intervention processes prescribed by the program processes prescribed by the program and and avoids those proscribed by the program.avoids those proscribed by the program.
CompetenceCompetence
Level of skillLevel of skill demonstrated by practitioner demonstrated by practitioner while delivering services or treatment to the while delivering services or treatment to the consumer.consumer.
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Fidelity AssessmentsFidelity Assessments
Fidelity assessments should include aspects Fidelity assessments should include aspects of :of :
Structure Structure
The framework for service delivery.The framework for service delivery.
Process Process
The way in which services are delivered.The way in which services are delivered.
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Methods to Develop a Fidelity Assessment
Requires the specification of a theory of change; Requires the specification of a theory of change; core components of intervention need to be core components of intervention need to be identified and measured.identified and measured.
Drawing from a specific program model with Drawing from a specific program model with proven efficacy and effectivenessproven efficacy and effectiveness
Gathering expert opinionGathering expert opinion
Survey of experts and/or literature Survey of experts and/or literature reviews to identify active ingredients or the core reviews to identify active ingredients or the core components of the program.components of the program.
Opinions of users and advocates regarding what Opinions of users and advocates regarding what works or site visits of diverse currently operating works or site visits of diverse currently operating programs. programs.
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
Measuring Intervention Fidelity
Most common methods:Most common methods:
Ratings by experts, based on project Ratings by experts, based on project documentation, client records, site documentation, client records, site observations, interviews and/or observations, interviews and/or videotaped sessions.videotaped sessions.
Surveys or interviews completed by the Surveys or interviews completed by the individuals delivering the services or individuals delivering the services or those who receive them.those who receive them.
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
For More InformationFor More InformationSandra F. Naoom, M.S.P.H.
919-636-2035
At the Frank Porter Graham Child Development Institute
University of North Carolina
Chapel Hill, NC
http://nirn.fpg.unc.edu/ http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/
Copyright © Dean L. Fixsen and Karen A. Blase, 2008
For More Information
Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).
Download all or part of the monograph at:
http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/
To order the monograph go to:
https://fmhi.pro-copy.com/