Toward a Learning Healthcare Delivery System: Leveraging Implementation, Improvement and

Delivery System Science to Improve Performance

November 11, 2015

Brian S. Mittman, PhDDept of Research and Evaluation, Kaiser Permanente Southern California

US Dept of Veterans Affairs Quality Enhancement Research Initiative (QUERI)UCLA CTSI Implementation and Improvement Science Initiative

Kaiser PermanenteRESEARCH

Defining a Learning Healthcare System

1. Outcomes and performance goals

2. Features and performance processes

Institute of Medicine’sCrossing the Quality Chasm (2001)

Safe: avoiding injuries Effective: services based on scientific knowledge; avoiding

underuse and overuse Patient-centered: responsive to individual patient

preferences, needs, and values Timely: reducing waits and delays Efficient: avoiding waste [value, affordability] Equitable: across gender, ethnicity, geography, SES

Commonwealth Fund’s Frameworkfor a High Performance Health System (2006)

Quality and Safety• the right health care, avoiding underuse, overuse and misuse• safe, reliable• coordinated• patient-centered: timely, excellent service, active and

informed patients

Access to Care• universal participation• financial protection, established benefits, affordable• equitable

Commonwealth Fund’s Frameworkfor a High Performance Health System (2006)

Efficient, High Value Care• efficient• right time, right setting• ongoing evaluation of new technologies; defined processes for

introduction, surveillance, reevaluation

System Capacity to Improve• investment in innovation and research • information infrastructure• effective educational system• rapid response to threats and disasters• culture of improvement• balance between autonomy and accountability

Summary of desired features

1. Safe and reliable2. High-quality, effective, evidence-based3. Patient-centered, excellent service4. Timely, accessible5. Efficient, cost-effective, high-value6. Equitable7. System is technologically advanced, research- and

improvement-oriented, balancing autonomy and accountability

Learning Healthcare Delivery System features

Explicit performance, improvement, learning goals Comprehensive performance monitoring against goals Explicit care management plans, policies, practices Active environmental scanning Explicit policies and processes for locating, vetting,

evaluating, refining, scaling/spreading external innovations “ “ “ for internal innovation, experimentation Supportive leadership, culture, training, resources (staff,

expertise, space, equipment, funds, etc.), rewards, etc.

Achieving learning and improvement: Role of improvement, implementation, delivery sciences?

1. Reliable evidence-based strategies (interventions) for delivering, improving care (FDA-approved, formulary-listed)

2. Strategies for working to improve care (e.g., PDSA/rapid-cycle improvement); analytical approaches and tools for monitoring and guiding improvement

3. Insights into barriers to change, requirements or conditions for improvement (environment, organization, team, ind’l)

4. Insights into the behavior of delivery systems and organizations, teams, clinicians and staff

What is implementation science?

1. Clinical research produces new evidence, innovation

2. Initial efforts to promote implementation

3. Measurement of rates of implementation – and implementation (quality) gaps

4. Research to develop and evaluate implementation programs* to increase adoption

* quality improvement programs, practice change programs (interventions)

Health benefits of research

Improved Health Processes, Outcomes

BasicScience

ClinicalResearch? ?

The Clinical Research Crisis

AAMC Clinical Research Summit: Clinical Research: A National Call to Action (Nov 1999)

IoM Clinical Research Roundtable (2000-2004)

UK Cooksey Report (2006), other US and non-US reports

Translational research

Improved Health Processes, Outcomes

BasicScience

ClinicalResearch

Type 1Translation

Type 2Translation

Translational research

BasicScience

Pre-Clinical/ Translational

Research

Type 2Translation

Implementation Research

Clinical Research

Type 1Translation

Improved Health Processes, Outcomes

Implementation research

BasicScience

Pre-Clinical/ Translational

Research

Implementation Research

Clinical Research

Improved Health Processes, Outcomes

Implementation science definition

Implementation research is the scientific study ofmethods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services.

It includes the study of influences on healthcare professional and organizational behavior.

Eccles and Mittman, 2006

Implementation science aims

1. Develop reliable strategies for improving health-related processes and outcomes; facilitate widespread adoption of these strategies

2. Produce insights and generalizable knowledge regarding implementation processes, barriers, facilitators, strategies

3. Develop, test and refine implementation theories and hypotheses; methods and measures

Clinical research vs. implementation research

Study typeStudy feature

Clinicalresearch

Implementation research

Aim: evaluate a / an … clinical intervention

implementation strategy

Typical intervention drug, procedure, therapy

clinician, organizational

practice change

Typical outcomessymptoms,

health outcomes,patient behavior

adoption, adherence,

fidelity

Typical unit of analysis, randomization patient clinician, team,

facility

US, international resources

NIH Conference on the Science of Dissemination and Implementation (2007 — 2015 )

NIH grant funding, review committee, training programs

Journals: Implementation Science, Translational Behavioral Medicine, special issues of general and specialty journals, new SIRC journal in development

NIH CTSAs (selected), PBRNs, ACTION, VA QUERI

Patient-Centered Outcomes Research Institute (PCORI), AAMC Research on Care Community (ROCC)

Knowledge Translation Canada, Kings College London Centre for Implementation Science, etc.

Local resources

Community Health, Health Behavior

Health Services, Management

GIM, Family/Prev Medicine, Subspec

Nursing, Dentistry, Psychology, Social Work, OT, PT, other allied

Psychology, Sociology, Anthropology, Political Science, Economics

Management, Education, Public Policy

Health Sciences: Main Campus:

Practice-focused research:Emerging models

VA QUERI

Academic Health System / School of Medicine “3I Institutes” (Improvement, Implementation, Innovation)

Integrated delivery system (Health Care Systems Research Network) embedded/partnership research:

– KPSC Care Improvement Research Team

– AcademyHealth Delivery System Science fellowship

Key features: joint governance, internal funding, negotiated scope, goals, standards (timeline, rigor)

The “Quality Chasm”

Institute of Medicine (1999, 2001)

Quality “report cards” (US, international)

Emergence and evolution of research interest

50+ years of research to identify causes and develop solutions to slow, uneven adoption of effective practices

• Changing physician behavior (1970s/80s: CME, reminders, incentives)

• Quality improvement, patient safety (1980s, 1990s, etc.)

• Implementation science (2000s to present)

• Findings, insights, recommendations are rich and valuable, yet difficult to apply

• VA, Kaiser and other systems have improved, but significant quality and performance gaps remain

The Tower of Babel problem

• Knowledge translation

• Translational research

• Research utilization, knowledge utilization

• Knowledge-to-action, knowledge transfer & exchange

• Technology transfer

• Dissemination research

• Quality improvement research

• T-1, T-2, T-3, T-4

• Etc.

Achieving learning and improvement: Contributions from improvement science

(QI research) vs. implementation science (1)

• QI often focuses on the “here and now” – immediate, local improvement needs via rapid-cycle, iterative improvement

• IS often attempts to develop, deploy and rigorously evaluate a fixed implementation strategy across multiple sites, emphasizing theory, contextual factors, (sometimes) mediators, moderators, mechanisms

• IS aims to develop generalizable knowledge

Achieving learning and improvement:Contributions from QI vs. implementation science (2)

• QI is pragmatic, improvement-oriented (often at the cost of limited confidence in interpretation and attribution and useful knowledge);

• IS is scientific, research/knowledge-oriented (often at the cost of improvement outcomes and useful knowledge)

• Neither has made much headway in achieving either goal

• QI often ignores contextual factors, fundamental insights into organizational/professional behavior, cross-site differences and implications for improvement success

• IS usually ignores heterogeneity and dominance of context over intervention main effects, and – too often – mediators, moderators, mechanisms

• Neither has made much headway in achieving either goal

Achieving learning and improvement:Contributions from QI vs. implementation science (3)

Achieving learning and improvement:Contributions from QI vs. implementation science (4)

• QI offers tools for persisting until improvement is achieved, driven by a desire to solve an identified quality problem

• IS offers theories, designs, methods, conceptual clarity for building from effectiveness/innovation work to implementation, to reap the benefits of innovation and research discovery and development

Achieving learning and improvement: Contributions from QI research and implementation science

• Despite some overlap, QI research and implementation science are largely complementary, and each could (should) learn and benefit from the other

Necessary conditions for practice change: insights from QI and implementation research

1. Valid, legitimate (accepted) evidence2. Evidence of deviations3. External expectations, interest (monitoring),

pressure4. Supportive professional norms5. Etiology of practices, deviations6. Information, evidence, education7. Feasible methods/systems

Necessary conditions for practice change

1. Valid, legitimate (accepted) evidence2. Evidence of deviations3. External expectations, interest (monitoring),

pressure4. Supportive professional norms5. Etiology of practices, deviations6. Information, evidence, education7. Feasible methods/systems

Implementation and QI in local settingsfactors contributing to success

Exceptional (non-routine, unsustainable, non-scalable) resources and support from central project team:

– site-by-site, individualized technical assistance– funding for new staff, services– recruitment, hiring, training, supervision, support for new

staff

Hawthorne effect (enhanced attention from monitoring, evaluation, external/internal interest)

Challenges to planned scale-upand spread

Lack of exceptional resources coupled with:

1. Features of innovations

2. Features of target adopters

3. Features of the environment

4. Features of innovation champions

5. Features of scale-up/spread strategies______________Source: WHO and ExpandNet, Practical Guidance for Scaling Up Health Innovations, 2009. http://expandnet.net/PDFs/WHO_ExpandNet_Practical_Guide_published.pdf

Barriers to progress?

Critiques and commentaries on the “state of the science” in implementation science often cite:

Lack of rigor; limited internal validity; too few RCTs

Limited external validity; too many RCTs (or too many flawed RCTs); use of “black box” evaluation approaches

Lack of theory; lack of appropriate theory

Too many theories; lack of guidance in using theory

Implementation and improvement problems and phenomena are extraordinarily complex (simple vs. complex vs. wicked problems)

An alternative (re-stated) hypothesis

Implementation and improvement science study phenomena characterized by:

Heterogeneity and variability of program (intervention) content across time and place

Heterogeneity of program implementation across time and place

Significant and variable contextual influences (leadership, culture, experience/capacity, staff/budget sufficiency)

Strong mediator effects (indirect impacts) and attenuation of effects

Weak main effects (other than for robust programs)

Studying complex social interventions:What is our goal?

Two very different questions

1. Does it work? Is it “effective”?Should it be approved? Included in the formulary?Should I use it?

2. How, why, when and where does it work?How should I use it?How do I make it work?

For many or most implementation strategies, Q1 is meaningless

Developing insights and guidance for implementation and improvement

How do I choose an appropriate implementation or improvement strategy given my context?

How do I implement (deploy) the strategy to increase effectiveness?

How do I adapt and customize the strategy to increase effectiveness (initially and over time)?

How do I modify (manage) the organization or setting to increase effectiveness (initially and over time)?

How, why, when and where does it work?