implementation science: research priorities and needs for the field ian d graham phd vice president...

Implementation Science: Research Priorities and Needs for the Field

Ian D Graham PhDVice President

Canadian Institutes of Health ResearchChicago, July 28th 2009

Cross QUERI Working Group onImplementation Science

Working in a living Lab

You have a unique opportunity to not only improve health and quality of life of your clients but

also to

further implementation (KT) science

• This can be done through 3 general strategies:– Move forward through retrospection and reflection – Make your tacit knowledge explicit through rigorous and

“infectious” reflection and documentation – Use conceptual frameworks

Why use a conceptual framework?

“It can scarcely be denied that the supreme goal of all theory is to make the irreducible basic elements as simple and as few as possible without having to surrender the adequate representation of a single datum of experience.”

Albert Einstein

Why use a conceptual framework?

• as with other forms of science, the success of a field is dependent on its theoretical foundation

• getting research into practice requires a systematic effort as the process is complex and occurs in the face of competing organizational and practice priorities

• hence researchers need to analyze and synthesize both the empirical and the theoretical underpinnings to be able to look inside the implementation “black box”

Why use a conceptual framework?• theory can be the driver of comparable and consistent

measurement/evaluation of implementation objectives• important for researchers to evaluate the utility of their

chosen theory including its internal and external validity – which components are critical, how should they be weighted

• The existing body of evidence about the effectiveness of different interventions suggests their impact is variable and effect sizes are generally moderate – thus evidence alone cannot guide implementation efforts

• We undertook a critical analysis of conceptual models of implementation as a means of understanding the theoretical underpinnings of implementation or what we at CIHR call knowledge translation.

• They have the basic purpose of focusing, ruling some things in as relevant and ruling others out due to their lesser importance.

Change theories/models fall into two basic kinds: Classical - models that describe change, but were

not specifically designed to cause change (e.g.PARIHS Promoting Action on Research Implementation in Health Services; Diffusion of Innovations) Planned – models that are intended to be used to guide or cause change (QUERI framework is a quality

improvement PAT)

…about conceptual frameworks

about conceptual frameworks

• Many of the PATs propose similar steps – can see that the QUERI framework addresses the action categories – the implementation “steps”

• The PATs are varied in their consideration of implemenation “factors” (covered in so called “classical” theories (of which there are many))

• Besides these macro and meso-level theories ,there are also many theories dealing with behaviour at the individual as well as organizational level (see chapter by Wensing et al. in “From Evidence to Practice” in particular, and the entire book, in general)

MonitorMonitorKnowledgeKnowledge

UseUse

SustainSustainKnowledgeKnowledge

UseUse

EvaluateEvaluateOutcomesOutcomes

AdaptAdaptKnowledgeKnowledge

to Local Contextto Local Context

AssessAssessBarriers/facilitatorsBarriers/facilitators

to to Knowledge UseKnowledge Use

Select, Tailor,Select, Tailor,ImplementImplement

InterventionsInterventions

Identify ProblemIdentify Problem

Identify, Review,Identify, Review,Select KnowledgeSelect Knowledge

Products/Products/ToolsTools

SynthesisSynthesis

Knowledge Knowledge InquiryInquiry

Tailo

ring

Kno

wle

dge

KNOWLEDGE CREATIONKNOWLEDGE CREATION

from: Graham et al: Lost in Knowledge Translation: Time for a Map?

http://www.jcehp.com/vol26/2601graham2006.pdf

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers/facilitators

to Knowledge Use

Select, Tailor,Implement

Interventions

Identify Problem

Identify, Review,Select Knowledge


SynthesisSynthesis


Tailo

ring

Kno

wle

dge


http://www.jcehp.com/vol26/2601graham2006.pdf

from: Graham et al:Lost in Knowledge Translation: Time for a Map?

Knowledge Inquiry

• studies using all types of study designs• primary research needs to be targeted to fill the known

gaps in our knowledge base • primary research needs to be solutions-based• Implementation Science:

– need research on effectiveness/impact of interventions and develop common methods of evaluation

• QUERI 4 Phase Pipeline: – Phase 1: single site pilot; – Phase 2: small scale/multi site implementation trial; – Phase 3: large scale/multi-region implementation trial; – Phase 4: system wide roll out

Products/Tools

Synthesis

Knowledge Inquiry

Tailo

ring

Kno

wle

dge

Synthesis

• need for synthesis to determine what we already know (or should know if we were to summarize the existing knowledge)

• can take the form of systematic reviews, meta analyses, realist reviews, etc.

• need to determine where there is a strong evidence base and move that evidence into action

Products/Tools

Synthesis

Knowledge Inquiry

Tailo

ring

Kno

wle

dge

Knowledge Products/Tools

• Algorithms• Decision rules• Decision aids• Guidelines• Measurement tools

Products/Tools

Synthesis

Knowledge Inquiry

Tailo

ring

Kno

wle

dge

MonitorMonitorKnowledgeKnowledge

UseUse

SustainSustainKnowledgeKnowledge

UseUse

EvaluateEvaluateOutcomesOutcomes

AdaptAdaptKnowledgeKnowledge

to Local Contextto Local Context

AssessAssessBarriers/facilitatorsBarriers/facilitators

to to Knowledge UseKnowledge Use

Select, Tailor,Select, Tailor,ImplementImplement

InterventionsInterventions

Identify ProblemIdentify Problem

Identify, Review,Identify, Review,Select KnowledgeSelect Knowledge


SynthesisSynthesis


Tailo

ring

Kno

wle

dge


Action Cycle(Application)

EITHER

Start with problem/issue concern and look for research to solve the problem

OR

Become aware of the research/guideline and assess whether current practice is in keeping with it

Identify Problem


Tailo

ring

Kno

wle

dge

Products/ Tools

Synthesis

Knowledge Inquiry

KNOWLEDGE CREATION

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


• To apply knowledge, one needs to contextualize or adapt it for local use– e.g. ADAPTE Collaboration (for guideline adaptation)

• How might the implementation plan need to be adapted for local use?

AdaptKnowledge to Local Context

SustainKnowledge

Use

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Products/

ToolsSynthesis

Knowledge Inquiry

KNOW

LEDGE CREATIO

N

MonitorKnowledge

Use

• Assess barriers and supports related to:– the knowledge (e.g. guideline)– the potential adopters– the practice setting– the implementation plan

• What might be the individual, team, and organizational barriers to uptake?

• Can the development process anticipate the barriers to use and address any of them?


toKnowledge Use

MonitorKnowledge

Use

SustainKnowledge

Use

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Products/

Tools

Synthesis

Know

ledge Inquiry

KN

OW

LED

GE

CR

EA

TIO

N

• Based on the barriers and supports identified, select or tailor implementation interventions

• e.g. Cochrane’s Effective Practice and Organization of Care review group source of synthesized info on effectiveness of KT interventions

http://www.epoc.cochrane.org/en/index.html


InterventionsSustain

KnowledgeUse

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring K

nowledge

Products/

Tools Synthesis

Knowledge Inquiry

KNOW

LEDGE CREATIO

N

MonitorKnowledge

Use

http://www.epoc.cochrane.org/en/index.html

• Assess the use or uptake of the knoweldge– Conceptual use (knowledge, attitudes, intentions) – Instrumental Use (behavioural), – symbolic knowledge use (to persuade)

• What “dose “ of the intervention is being delivered?

MonitorKnowledge

Use

MonitorKnowledge

Use

SustainKnowledge

Use

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Know

ledge

Products/ Tools

Synthesis

Knowledge Inquiry

KNOWLEDGE CREATION

• Determine impact of the intervention– Client health and other outcomes (eg. symptoms,

morbidity, HRQoL, mortality, satisfaction)– Provider outcomes (eg. satisfaction)– System/organization outcomes (eg. Wait times,

expenditures, readmissions)– Unintended impacts

EvaluateOutcomes

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Pro

duct

s/

Too

ls

Syn

thes

is

Kno

wle

dge

Inqu

iryK

NO

WLE

DG

E C

RE

AT

ION

• What is needed to sustain ongoing use?• What ongoing monitoring of knowledge use and

impacts needed?• How long are the skills/knowledge maintained?

SustainKnowledge

Use

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailor

ing K

nowled

ge

Pro

duct

s/ T

ools

Syn

thes

is

Kno

wle

dge

Inqu

iry

KN

OW

LED

GE

CR

EA

TIO

N

CIHR:Two kinds of KTCIHR:Two kinds of KT

End of grant KT

Integrated KT

The researcher develops and implements a plan for making knowledge users aware of the knowledge generated through a research project

Research approaches that engage potential knowledge-users as partners in the research process. requires a collaborative or participatory approach to research that is action oriented and is solutions and impact focused (Mode 2).

What is integrated KT?

• a way of doing research• collaborative, participatory, action-oriented, community

based research, co-production of knowledge, mode 2 knowledge production

• involves engaging and integrating knowledge users into the research process

• Knowledge users can be:– Policy- and decision-makers from the community to the

federal level, researchers, the public, industry, clinicians, the media

– Investigators from different disciplines, teams, countries

In addition, knowledge users and researchers (knowledge creators) can work together to:

shape the research questionsdecide on the methodology interpret the study findings and craft messaging

around themhelp with data collection and tools developmentmove the research results into practicewidespread dissemination and application

What is integrated KT?

IKT and QUERI

• QUERI’s action-oriented approach that meaningfully engages clinicians, managers, patients/clients, and researchers in research-driven initiatives to improve quality = IKT (Mode 2 knowledge production)

• IS research agenda:– What has been learned so far about IKT at QUERI?– What’s involved in doing QI this way?– How is it working (how effective is it)?– How can it work better?– Strategies and tools to engage knowledge users– Guidelines/criteria for good Mode 2 knowledge production

(IKT)

Comparing the knowledge to action

cycle with the QUERI model

Step 1: Select conditions per patient populations associated with high risk of disease and/or disability and/or burden of illness for veterans

1A. Identify and prioritize (via a formal ranking procedure)1B. Identify high-priority clinical practices and outcomes

within a selected condition

Step 3: Measure and diagnose quality and performance gaps

3A. Measure existing practice patterns and outcomes across VA and identify variations from evidence-based practices ("quality/performance gaps")

3B. Identify determinants of current practices3C. Diagnose quality/performance gaps

Identify Problem


Tailo

ring

Kno

wle

dge

Products/ Tools

Synthesis

Knowledge Inquiry

KNOWLEDGE CREATION

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


-------------------------------------------------------------------------------------------------

Step 2: Identify evidence-based guidelines, recommendations and best practices

2A. Identify evidence-based clinical practice guidelines2B. Identify evidence-based clinical recommendations2C. Identify evidence-based clinical practices

Identify Problem


Tailo

ring

Kno

wle

dge

Products/ Tools

Synthesis

Knowledge Inquiry

KNOWLEDGE CREATION

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem



SustainKnowledge

Use

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Products/

ToolsSynthesis

Knowledge Inquiry

KNOW

LEDGE CREATIO

N

MonitorKnowledge

Use

Step 4: Implement improvement programs4A. Identify improvement/implementation strategies, programs and program components or tools

4B. Develop or adapt improvement/implementation 4B. Develop or adapt improvement/implementation strategies, programs and program components or strategies, programs and program components or toolstools

4C. Implement improvement/implementation strategies/programs to address quality gaps


SustainKnowledge

Use

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Products/

ToolsSynthesis

Knowledge Inquiry

KNOW

LEDGE CREATIO

N

MonitorKnowledge

Use

Step 4: Implement improvement programs

4B. Develop or adapt improvement/implementation 4B. Develop or adapt improvement/implementation strategies, programs and program components or strategies, programs and program components or toolstools•IS Research Agenda:IS Research Agenda:

•Most efficient and useful methods for adapting Most efficient and useful methods for adapting guidelines/research to settingsguidelines/research to settings•How to develop local capacity for adaptation and How to develop local capacity for adaptation and sustainable process for adaptationsustainable process for adaptation


3A. Measure existing practice patterns and outcomes across VA and identify variations from evidence-based practices ("quality/performance gaps")

3B. Identify determinants of current practices3C. Diagnose quality/performance gaps

3D. Identify barriers and facilitators to improvement3D. Identify barriers and facilitators to improvement


toKnowledge Use

MonitorKnowledge

Use

SustainKnowledge

Use

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Products/

Tools

Synthesis

Know

ledge Inquiry

KN

OW

LED

GE

CR

EA

TIO

N


3D. Identify barriers and facilitators to improvement3D. Identify barriers and facilitators to improvement• IS research agenda:

– Determinants of research use (role of unit/org culture, leadership)– Taxonomy for barriers and facilitators (individual, team,

organizational)– Validated measures to identify barriers and facilitators– Most efficient methods of identifying barriers and facilitators (e.g.

surveys vs interviews vs key informants)


toKnowledge Use

MonitorKnowledge

Use

SustainKnowledge

Use

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Products/

Tools

Synthesis

Know

ledge Inquiry

KN

OW

LED

GE

CR

EA

TIO

N

Step 4: Implement improvement programs4A. Identify improvement/implementation

strategies, programs and program components or tools

4B. Develop or adapt improvement/implementation strategies, programs and program components or tools

4C. Implement improvement/implementation strategies/programs to address quality gaps



KnowledgeUse

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring K

nowledge

Products/

Tools Synthesis

Knowledge Inquiry

KNOW

LEDGE CREATIO

N

MonitorKnowledge

Use

Step 4: Implement improvement programs4A. Identify improvement/implementation strategies, programs and

program components or tools4C. Implement improvement/implementation strategies/programs to

address quality gaps

• IS research agenda:– Effectiveness of implementation strategies (what works

for whom, under what conditions, with what evidence (guidelines)?) Methods to determine dose of strategies

– Strategies directed at individuals (e.g. psychological) vs social setting/organization (e.g. sociological, organizational) (health system interventions key)



KnowledgeUse

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring K

nowledge

Products/

Tools Synthesis

Knowledge Inquiry

KNOW

LEDGE CREATIO

N

MonitorKnowledge

Use

Step 5/6: Evaluate improvement programs

5. Assess improvement program feasibility, 5. Assess improvement program feasibility, implementation and impacts on patient, family implementation and impacts on patient, family and healthcare system processes and outcomesand healthcare system processes and outcomes

6. Assess improvement program impacts on health related quality of life (HRQOL)

MonitorKnowledge

Use

MonitorKnowledge

Use

SustainKnowledge

Use

Evaluateoutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Know

ledge

Products/ Tools

Synthesis

Knowledge Inquiry

KNOWLEDGE CREATION


5. Assess improvement program feasibility, implementation and impacts on patient, family and healthcare system processes and outcomes


EvaluateOutcomes

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Pro

duct

s/

Too

ls

Syn

thes

is

Kno

wle

dge

Inqu

iryK

NO

WLE

DG

E C

RE

AT

ION


5. Assess improvement program feasibility, implementation and impacts on patient, family and healthcare system processes and outcomes


• IS research agenda:– Validated measures of knowledge use and impact

EvaluateOutcomes

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailoring Knowledge

Pro

duct

s/

Too

ls

Syn

thes

is

Kno

wle

dge

Inqu

iryK

NO

WLE

DG

E C

RE

AT

ION

<!> the sound of one hand clapping

• IS research agenda:– Determinants of sustained knowledge use

(organizational, team, individual)– Methods to measure ongoing use and impact– Sustainability of implementation strategies overtime

and their impact– Creating sustainable research use organizations and

cultures

SustainKnowledge

Use

MonitorKnowledge

Use

SustainKnowledge

Use

EvaluateOutcomes

AdaptKnowledge

to Local Context

AssessBarriers to

Knowledge Use


Interventions

Identify Problem


Tailor

ing K

nowled

ge

Pro

duct

s/ T

ools

Syn

thes

is

Kno

wle

dge

Inqu

iry

KN

OW

LED

GE

CR

EA

TIO

N

Finding your way through the implementation maze

Welcome to QUERI!This way for useful

advice

How to move the implementation science research agenda forward

• You have the QUERI Framework (PAT) and QUERI Pipeline– Review and refine the frameworks– Consciously categorize all activities by these

frameworks to allow better understanding of projects

• You have selected a classical theory to define how you examine/approach the factors (PARIHS, for example)– Theory testing


• Theories range from the simple to the complex• Not practical nor feasible to control and/or measure

every possible variable nor to test every construct or hypothesis

• Need to do enough ground work to understand the implementation specific context and to get a sense of what the key change objectives are important to the knowledge-users in that context

• Need to balance need for context-specific tailoring with need for replicability and commonality of measurement tools


• View all QI efforts as opportunities to advance implementation science– Be reflective: document processes and

experiences (the science and the art)– What are you learning? – Turn tacit knowledge about implementation and

implementation research into explicit knowledge- i.e. how to conduct implementation research but also how to implement


• Develop and test measures (barriers assessment, knowledge use, impact, sustainability)

• Encourage use of common measures (or batteries of measures) to allow comparison across studies

How to choose the appropriate intervention

• Link specific interventions to the barriers– This is an art, not a science – If barrier is knowledge or confidence, then self

efficacy theory can be used, if it is lack of awareness, then the transtheoretical model could be used etc. Test methods of tailoring strategies to barriers

– involving a large number of potential knowledge-users in this process could contribute to a successful intervention (but this is an empirical question )

How to choose the appropriate intervention

• Decide if you need a single or multiple/complex interventions

• With multiple interventions – are you targeting the same or different barriers?

• Are the interventions targeted to different levels: individual, team, organization?


There are also many theories dealing with behaviour at the individual as well as organizational level (see chapter by Wensing et al. in “From Evidence to Practice” in particular, and the entire book, in general)

• Whenever you are combining the QUERI framework with other theories, consider documenting your experiences so as to advance understanding of how useful the model is and to provide information to others who are attempting a similar project– Have you been able to derive and test hypotheses and

propositions based on the theory?– Does it explain successes and failures?– Does it suggest or provide ways to measure/evaluate the

hypotheses?– Does it adequately capture the critical components (the

active ingredients) of the implementation process in your project?

Back to the idea of working in a living Lab

You have a unique opportunity to not only improve health and quality of life of your clients but also to

further implementation (KT) science

• This can be done through 3 general strategies:– Move forward through retrospection and reflection

– Make your tacit knowledge explicit through rigorous and “infectious” reflection and documentation

– Use conceptual frameworks

working in a living Lab

You have seen the handout with implementation science goals listed in recent publications

Think about how your work can push the implementation science (KT) agenda

Relationships seem to be a key driver of implementation efforts – can this be studied and documented as well practiced?

Context is seen as important: what are the key features to either sink an implementation effort or successfully launch one?


We need to be able to monitor knowledge use, evaluate outcomes and sustain change

To do this effectively, transparently and in a manner that is reproducible, we need appropriate measurement tools

It is not helpful if everyone makes up their own every time they need one

Identify (adapt or develop) and use psychometrically validated tools

working in a living LabSustainability is a key issue that needs basic research

(how is it defined, how often and under what circumstances does it happen) as well as enabling research (what can be done to ensure sustainability – in a variety of contexts and settings)

Think back to past implementation successes and failures: how can theory be harnessed to differentiate them?

As implementation scientists, we need to work reflexively: using what we know about knowledge translation to inform and guide our own work


• Capacity development– Implementation researchers– Among service providers to effectively

facilitate implementation

• Ensuring relevance with service providers- making QUERI indispensible


• QUERI is at the cutting edge of Implementation Science

• What does QUERI need to do to stay at the cutting edge in the future?

Knowledge, if it does not determine action, is dead to us

Plotinus (Roman philosopher 205AD-270AD)

[email protected]

Thank you for your attentionMerci

For more information, visit our web page:http://www.cihr-irsc.gc.ca/e/29418.html http://www.cihr-irsc.gc.ca/f/29418.html

implementation science: research priorities and needs for the field ian d graham phd vice president...

Documents

implementation science

research implementation

implementation efforts

implementation kt science

implementation black

general slide

monitor knowledge use

cause change queri framework