jeff alexander regenstrief conference slides
DESCRIPTION
The Science and Practice of Implementation : Are We headed Down the Right Path?TRANSCRIPT
The Science and Practice of The Science and Practice of Implementation : Are We Implementation : Are We
headed Down the Right Path?headed Down the Right Path?
10th Biennial Regenstrief Conference 10th Biennial Regenstrief Conference Emerging Perspectives on Transformational Emerging Perspectives on Transformational
Change in Healthcare SystemsChange in Healthcare Systems OctoberOctober 2 2 –– October October 4, 2007.4, 2007.
Jeff Alexander, Ph.D.Department of Health Management and Policy
University of Michigan School of Public Health
Goals
Problems with Quality Improvement Research- why it’s not contributing to systems change
State of the art- Implementation Research- good, bad, and ugly
Modest proposals for advancing the science of implementation and the usefulness of QI research
State of the Art – QI Research
Single organizational samples Opportunistic not systematic Short duration studies No replication of studies No explicit consideration of context No explicit consideration of
implementation
Bottom line:
We don’t know what works, when it works, or where it works
BEYOND THE LINEAR MODEL
Basic Research
Clinical Trial
(Efficacy)
TreatmentDevelopment
EffectivenessTrial
TreatmentDeployment
Problems with the Linear Model of Implementation
Little on causal pathways & nested interconnected structures and activities
Little influence of OT & OB in QI studies
RCT thinking: control context away
RCTs as the Gold Standard?
Great for testing efficacy of molar interventions
Not so great for assessing: Process related phenomena Complex interactions among program
components Contextual effects implementation
BackgroundBackground
Health care provided in organizational Health care provided in organizational contextcontext
behavior of clinicians influenced by the behavior of clinicians influenced by the organizations in which they workorganizations in which they work
recognition of the interconnections among recognition of the interconnections among components of organizations (clinical components of organizations (clinical teams function within hospitals, interact teams function within hospitals, interact with other clinical teams, support systems with other clinical teams, support systems - embeddedness- embeddedness
Implementation-State of the Art
Emerging lists of “best practices” Continued assumption, encouraged
by funding streams, of linear development of interventions
Anecdotal information on implementation
Some efforts to produce models, theories to test implementation
Implementation: the influence of content, context, and process
Implementation
Content Process•Opinion leaders, change champion
•Systemic processes (e.g., supervisory practices, quality improvement)
•Organizational learning
•Triability
•Innovation type
•Evidence interpretation and packaging
Internal:
•Organizational culture
•Organizational structure
•Practice setting characteristics
•Local stakeholders (e.g., attitudes and behaviors)
•Resources
External:
•Networks
•regulation
•Economic (e.g., reimbursement)
•Competition
Context
Management Support:Management
communicates a rationale and priority
Klein and Sora Model
Implementation Effectiveness:Consistency and
quality of innovation use
Implementation Effectiveness:Consistency and
quality of innovation use
Innovation-Values Fit:The perceived fit
between the end-user's values and the
innovation
Champion(s):Champion(s) promotes
the innovation with targeted org members and/or management
Financial Resource Availability:
Resources are made available to support
implementation policies and practices
Implementation Climate:
The innovation is perceived as an
organizational priority by targeted end users
Implementation Policies and Practices:
Formal organizational actions ensure user
skills, create incentives and/or identify and
address barriers to use
Why Context Matters
Context may affect implementation directly
Context may moderate the relationship between an innovation and outcomes of interest
Context may establish the external validity of both implementation and QI research
Problems with Implementation Problems with Implementation Context Measurement and Context Measurement and
AnalysesAnalyses assigning the same group value to all assigning the same group value to all
members of a groupmembers of a group aggregating individual outcomes to aggregating individual outcomes to
the group levelthe group level Separate analyses of organizational Separate analyses of organizational
and individual phenomenaand individual phenomena
Advantages of Multilevel Advantages of Multilevel DesignsDesigns
statistically efficient estimates of statistically efficient estimates of regression coefficients regression coefficients
Use of clustering information Use of clustering information provides correct standard errors, provides correct standard errors, confidence intervals and significance confidence intervals and significance teststests
Allows for uneven assessments and Allows for uneven assessments and different program tenures (for different program tenures (for longitudinal studies)longitudinal studies)
Advantages of MLDAdvantages of MLD
Measurement at any of the levels of Measurement at any of the levels of a hierarchy enables examination of a hierarchy enables examination of whether differences in average whether differences in average outcomes between organizations are outcomes between organizations are explained by factors such as explained by factors such as organizational practices/structuresorganizational practices/structures, , or other characteristics of individual or other characteristics of individual patients or providerspatients or providers
0
0.5
1
1.5
2
2.5
3
entry 1 year 5 years
ADL
impa
irmen
t
low participation
high participation
Potential applications of Potential applications of MLDMLD
Effects of organizational infrastructure Effects of organizational infrastructure on implementation in micro teamson implementation in micro teams
Effects of org. culture on individual Effects of org. culture on individual provider attitudes and behavior (e.g. provider attitudes and behavior (e.g. physician use of clinical guidelines) physician use of clinical guidelines)
Translational researchTranslational research Effects of micro-team structure and Effects of micro-team structure and
process on patient outcomesprocess on patient outcomes
Issues with Multilevel Issues with Multilevel AnalysisAnalysis
Data requirementsData requirements Statistical powerStatistical power Analysis and interpretation issuesAnalysis and interpretation issues
Multi-method DesignsMulti-method Designs
Quantitative-QualitativeQuantitative-Qualitative RCT-case studyRCT-case study Process study-outcome studyProcess study-outcome study
Sustainability- long term studiesSustainability- long term studies
Life Cycle of Quality Improvement
Time
90
80
70
60
50
40
30
20
10 0
Adoption Rate
Implementation
Diffusion
Plateau
Institutionalization
Science of Complexity
Assumptions regarding interactions among components or “agents” of the system Heterogeneity- agents differ in important
characteristics (e.g. preferences) Dynamic-agents change, how system
changes are non linear, chaotic Feedback- change often results from
feedback that agents receive from their own behavior
Complexity Science
Organization- agents organized into groups or hierarchies that influence how system evolves over time
Emergent behavior- what results from the actions and interaction of individual agents
Engaged Scholarship
Theory
Solution
Model
Reality Proble
m F
orm
ulatio
n Theory Building
Resea
rch
DesignProblem
Solving
Describe Problem/Issue- visit & study it- map & diagnose itFormulate the Question- from users’ perspective?Criterion - Relevance
Answers & Arguments- plausible alternatives-- clarify context- identify key variations- cross levels of abstractionCriterion - Validity
Obtain the Evidence- case/field/experimental study- unit selection & sampling- measurement & observation- data analysisCriterion - Truth
Application & Implementation- knowledge for what? who?- for science & profession - apply findings to problem- develop implementation planCriterion - Impact
Capacity Building for Implementation Research
Data Funding Multi-disciplinary teams Make implementation part of the
intervention Bring in users of intervention/innovation Long term studies Basic research on implementation
Other questions
What aspects of care are modular and what aspects are inherently interdependent?
To what extent can one element of a system be altered without consequences to other elements?
Should intervention content and context dictate implementation process?