medical thinking or clinical action ?
DESCRIPTION
Medical thinking or clinical action ?. Dr Jeremy Wyatt FRCP DM Professor of health informatics, University of Dundee, Scotland [email protected] Acknowledgements: Prof Susan Michie, UCL. The problem. The NHS costs £80Bn per annum; there are severe workforce pressures - PowerPoint PPT PresentationTRANSCRIPT
Medical thinking or clinical action ?
Dr Jeremy Wyatt FRCP DM
Professor of health informatics, University of Dundee, Scotland
Acknowledgements: Prof Susan Michie, UCL
The problem
• The NHS costs £80Bn per annum; there are severe workforce pressures
• For some tests and therapies, we know enough about what helps patients to recommend that their use should be reduced or increased
• Despite this evidence, there is much geographical variation in clinical practice and patient outcomes
How can we narrow the gap between what clinicians know and what they do ?
What evidence do we have ?
Source: 2329 therapies reviewed for Clinical Evidence issue 12, 2005
Known to benefit patients
15%
Likely to do nothing / harm
4%
Likely to benefit21%
Benefit / harm trade off
7%
Unlikely to benefit
5%
Not known48%
Why medical thinking ?
• In the UK in 2006, more prescribers are non-medical than medical: nurses, pharmacists
• Patients & carers responsible for key health related decisions: – Are my symptoms serious ?
– Who to go to ?
– Do I believe them ?
– Shall I take this therapy ?
– Is it better now ?
• Strong pressures toward multi-skilled clinical practitioners, shared decision making, team based care
• Clinical, not medical, is what matters
Why clinical thinking ?
• Mismatches between what people know, remember, think, decide, intend, say and do
• Limited skills, confidence, self efficacy to put decision into action
• Pressures from peers, patients, NHS Trusts, NICE…• Action slips• Limited resources to carry out action: staff, time,
equipment…
Result: gulf between the good intentions resulting from clinical thinking and actual clinical actions
Practice guidelines
Evidence
Practice guideline
recommendation 1Expert opinions
recommendation 2
• Cost of tests, drugs
• Health system reality
• Society’s values• Lobbying from
patients, industry
Constraints
recommendation 3
What can we do with guidelines ?
Guideline
Clinician
Library
provide access
searches
Newsletter
disseminate
receives
Innovation method
use as basis of
participates in
Clinical practice innovation
Aim: to narrow the gap between what we know and do
Synonyms: implementation of research, behaviour change, getting research into practice, change management
At least 63 methods available, including: – Paper / computer reminders– Audit and feedback– Patient information leaflets– Decision support systems– Outreach visits– Opinion leaders...
Impact of outreach visits on care given to 4500 pregnant women
25 Eligible obstetric units
Baseline data collection
Randomisation
Follow-up data collection
Follow-up data collection
CCPC: Cochrane module on pregnancy & childbirth (Wyatt, BMJ 1998)
Outreach visit:•Guideline feedback•Discuss EBM, give video, CCPC, train
•Discuss innovation methods, give slides
control units
outreach units
Control clinicians
Innovation clinicians
Difference in practices
Difference in outcomes
Recommendation
Innovation method
Summarising evidence on innovation methods
Evidence
about
innovation
methods
Systematic review of innovation studies
Control clinicians
Innovation clinicians
Difference in practices
Difference in outcomes
Recommendation
Innovation method
Control clinicians
Innovation clinicians
Difference in practices
Difference in outcomes
Recommendation
Innovation method
Example: review of manual paper reminders
Definition: reminder, decision support or audit & feedback ?
Finding studies: 324, spread across 101 journals
Study quality: only 22 RCTs of 82 relevant studies of manual reminders
Other issues:– 17 studies from USA (largely fee-for-service)– 3 “positive” studies had unit of analysis errors & 1 showed 2%
change - so 10 clinically positive studies– Poor reporting – able to examine only 5 of 10 effect modifiers of
interest– Success rate varied by reminder type and targeted clinical
practice; need meta-regression
(Wyatt et al 2001)
Cochrane EPOC review group
Founded 1994 by JW as “Cochrane Collaboration on Behaviour Change” (!)
Now “Effective Practice & Organisation of Care” group
Base in Canada: Google “Cochrane epoc”
Editors: Jeremy Grimshaw, Andy Oxman, Merrick Zwarenstein, Lisa Bero
Output: 26 completed reviews, further 17 in progress
How likely are disseminated guidelines etc. to improve clinical practice ?
Systematic reviews Improved practices
Evidence summaries(Oxman CMAJ 1995)
11% 1/9
Printed educational materials(Freemantle CL 1997)
14% 2/14
Educational materials(Davis JAMA 1995)
36% 4/11
Guidelines in general practice(Wensing BJGP 1998)
11% 2/18
Average (duplicate studies): 17% 9/52
How likely are lectures to improve practice ?
Improved practices
Lectures only 0 0/4
Interactive sessions 67% 4/6
Mixed interactive / lecture sessions 71% 5/7
Single session 28% 2/7
Multiple sessions 70% 7/10
Students choose session content 80% 4/5
Overall 53% 9/17
(Davis, JAMA 1999)
How likely are clinical innovation methods to improve practice ?
Improved practices Paper reminders Wyatt 2001 46% 10/22
Audit & feedback Thomson CL ‘98 62% 24/39
Computer decision support Garg JAMA ‘05 64% 62/97
Patient materials Davis JAMA ‘95 78% 7/9
Opinion leaders Thomson CL ‘98 86% 6/7
Outreach visits Thomson CL ‘98 94% 15/16
Computer drug dosing Walton BMJ ‘99 100% 17/17
GP computing Sullivan BMJ ‘95 100% 21/21
Telemedicine Currell CL 2000 ? ?/7
Overall 73% 143/196
How likely are different methods to improve patient outcomes ?
Improved outcomes
Outreach visits (94%) Thomson CL ‘98 0% 0/1
Audit & feedback (62%) Thomson CL ‘98 18% 1/6
Paper reminders (46%) Wyatt 2001 18% 1/6
Telemedicine (?) Currell CL 2000 20% 1/5
GP computing (100%) Sullivan BMJ ‘95 33% 1/3
Opinion leaders (86%) Thomson CL ‘98 33% 1/3
Computer decision support (66%) Garg JAMA ‘05 13% 7/52
Computer drug dosing (100%) Walton BMJ ‘99 83% 5/6
Patient materials (78%) Davis JAMA ‘95 ? ?/9
Overall (73%) 19% 17/91
Low numbers !
Which clinical practices are easiest to improve ?
Improved practices
Procedural skills 25% 1/4
Diagnosis 50% 2/4
Disease management 55% 32/58
Ordering tests, procedures 71% 17/24
Preventive care 74% 40/54
Prescribing 80% 11/14
Overall 65% 103/158
Davis D. JAMA 1995; 274:700-5
The gulf between recommendations and outcomes
Improved clinical practice
Improved patient outcome
Guideline recommendation
Clinician wants to improve their practice
motivation, incentives
Clinician able to improve their practice
resources, skills, support
commitment, good memory
Clinician knows recommendation
knowledge access, dissemination
Matching innovation methods to barriers
Improved patient outcome
Clinician knows recommendation
knowledge access, dissemination
Clinician wants to improve their practice
motivation, incentives
Clinician able to improve their practice
resources, skills, support
commitment, good memory
Improved clinical practice
Not knowing Outreach visit, TM
Not caring Audit and feedback
No drugs, equipment Provide drugs, equipment
No skills, support Train with opinion leader, TM
Fear of consequences Target peer group, patients
Forgetting, action slips Reminder, decision support
Guideline recommendation
Barrier to change Innovation method
Should we bother with barriers ?
Improved practices
Innovators ignored barriers 42% 5/12
Checked literature 53% 18/34
Obtained local concensus 58% 26/45
Checked national guideline 61% 25/41
Local study to check barriers 90% 25/28
Davis D. JAMA 1995; 274:700-5
How long might innovation take ?
Time
Uptake %
100%
0
Rogers EM. The diffusion of innovations. New York Free Press 1993
innovators
early adopters
late adopters
laggards
What might make change more likely ?
The change is compatible with participant needs, norms, beliefs
The change is relevant to user’s work, provides an advantage for participants
Benefits can be easily observed, limited risk
The change is simple, can be broken down into steps, has a core with fuzzy boundaries
The change can be re-invented locally
The change is easy to try out – no infrastructure needed
All necessary knowledge or support available
(Rogers ’93, Greenhalgh Millbank Q 2005)
Relevant theories
1. PRECEDE (Green L, 1988):• Predispose the person to change• Enable the change• Reinforce the change
2. Theory of planned behaviour (Ajzen I, 1991)
Theory of planned behaviour
Human action is guided by three considerations:
• Behavioural beliefs about the likely outcomes of the behaviour and evaluations of these outcomes
• Normative beliefs about the expectations of others and motivation to comply with these
• Control beliefs about factors that may facilitate or impede performance of the behaviour and perceived behavioural control
In combination, these lead to intention to perform the behaviour in question
Intention assumed to be the immediate antecedent of behaviour. Given a sufficient degree of actual control over the behaviour, people
are expected to carry out intentions when the opportunity arises.
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50, 179-211.
Relevant psychological constructs
Consensus study of 62 psychologists etc.:1. Knowledge2. Skills3. Professional role and identity4. Beliefs about capabilities 5. Beliefs about consequences6. Motivation and goals7. Memory, attention and decision processes8. Environmental context and resources9. Social influences 10. Emotion11. Action plans12. Nature of the behaviour
Michie S, Johnston M, et al (2005) Making psychological theory useful for implementing evidence based practice: a consensus approach, Quality and Safety in Health Care, 14, 26-33
Wording of recommendations
The challenge is to ensure: • Comprehension – understanding what needs be done• Recall – remembering what to do in the relevant context• Planning of this behaviour• Performance of behaviour
Specificity of instructions is associated with attaining goals (Locke and Latham, 1990), so specify:
• Who should do • What• How• Where, and• WhenUse “if-then" plans and active verbs (do) Avoid general exhortations (should), passive descriptions (may)
Michie, S. and Johnston, M. Changing clinical behaviour by making guidelines specific. BMJ 2004, 328: 343–5
Evidence about specific recommendation wording
Grol’s study: 61 GPs & 47 recommendations from 10 national Dutch clinical guidelines
GPs followed recommendations: – on 2/3 of occasions when it was concrete and specific
– on 1/3 of occasions when it was vague and non-specific
NB. Specificity accounted for only 17% of the variance
Grol et al. Attributes of clinical guidelines that influence use of guidelines in general practice: observational study. BMJ 1998, 317:858-61
Some research issuesBarriers to change:
– Are the methods to elicit them reliable, valid ?– Are some barriers more important than others ?– Are barriers generic, or must we always tailor innovation ?
Practice innovation methods:– Is there evidence to match methods to barriers ?– How can psychological theory help in designing methods ?– Are some innovation methods more effective in specific:
• combinations ?• clinical practices or settings ?• professional groups ?
General: – Can we develop a valid “intention to implement” scale ?– Can an “innovation toolkit” & web site help clinicians change ?
Make the change
UCL KMC clinical practice innovation model
Isinnovationneeded ?
Audit local practice
Analyse barriers to change
Select & apply innovation method(s)
No
Choose a clinical practice
recommendation
Identify & engage all participants
Yes
Wait for a while
Loop A
Loop B
Examples of participants
GP prescribing: GPs, patients, pharmacists, nursing home staff…
Lab test ordering: junior doctors, senior doctors, medical schools, lab staff, patients, phlebotomists…
Cardiovascular risk reduction: patients, friends, relatives, manager of workplace / private gym / pub, local council, regional government, food industry…
Some practical conclusions
1. Guidelines do not themselves change practice
2. An innovation programme is needed:– Choose a clinical practice from the guideline that really does improve (or
worsen) patient outcomes– Obtain high-level support for an innovation programme
3. Implement the programme:– Carry out a careful local audit– Identify all participants (including opinion leaders)– Search for barriers to change– Select appropriate innovation method(s)– Monitor progress; consider a rigorous trial
4. Collaboration with psychologists, management scientists & ethnographers is likely to be useful
Some research questions
• How to use psychological theory to design and evaluate innovation methods ?
• Do different theories apply to increasing and decreasing actions ?
• Are barriers generic and enduring or person and episode specific ?
• How to validate theories eg. functional imaging• How much clinical variation depends on individual
cognition vs. team dynamics / environment ?
Does diagnosis matter ?
• At least 1/3 of encounters associated with long term illness – diagnosis made years ago
• Most advances in 20th century associated with quicker, more accurate diagnostic tests
• Challenging diagnoses rare in routine clinical practice• Challenges for health services are:
– which test in what order
– assembling patient data from multiple sources
– test interpretation
– monitoring of long term illness
– improving teamwork (work force)
– shifting services closer to home – self testing
One innovation method, or many ?
Improved practices
One method 60% 49/81
Two methods 64% 25/39
Three or more 79% 31/39
a) Indirect, between-study comparison (Davis, 1995):
b) Direct, within-study comparisons (Wensing, 1994): Multi-part interventions led to a larger effect in general practice than a single intervention in 4 / 10 RCTs
How long does the impact last ?
Thomson O’Brien. CL 1999
Few RCTs follow-up after audit & feedback stop:
3 months: both groups improved test usage (Martin A ‘80)
12 months: generic prescribing declined, but still better than control
group (Gehlbach ‘84)
14 months: improved management of cystitis (Norton P ‘85)
Conclusion: “There is insufficient data to clarify when the effects are most
likely to deteriorate after feedback stops”
How much does innovation cost and save ?
Cost of outreach visit: 1400 euro (1995 prices; Wyatt BMJ ‘98)
Cost to deliver paper reminders (Wyatt unpub.): – measured in 4 studies
– ranged from 10c to 75 euro per patient
– cost one sixth of the cost of pt. invitation letters for cervical screening
Savings from paper reminders: – cut inflation in asthma treatment costs to one third
– saved 1100 euro per inpatient (earlier discharge)
What might help guidelines succeed ?
Opinion survey of 1500 US internists - features they claim would cause them to comply (Hayward JGIM ‘96):
– Guidelines as short pamphlet: 86% in favour– Summary of supporting evidence: 85%– Benefits quantified: 77%– Endorsed by respected colleague (72%) or major organisation
(69%)
Study of actual practice - guideline features & actual use by 61 Dutch GPs (12 features, 12900 decisions - Grol BMJ ‘98):
– Positive correlation: recommendation was specific, uncontroversial, required no change in existing routine
– However, these accounted for only 17% of the variance
Systematic reviews of innovation studies
Special problems:– Innovation methods poorly defined, indexed, reported– Studies heterogeneous: different clinicians, settings, practices...– Few direct comparisons within the same study– Study designs poor, with bias and confounding (co-interventions,
Hawthorne Effect, contamination…)– Multiple measures of clinical practice - often subjective
Consequences:– Hard to identify studies, few studies eligible– Qualitative review (vote counting) usually more appropriate than
quantitative (meta-analysis)– Meta regression rarely possible: too many variables, too few
studies
From knowledge to outcome
External knowledge
Internalised knowledge
MotivationPerceived self efficacyClinical
decision
Opportunity
Distraction
Forgetting
Action slipsCognitive biases
Framing effects
Perception
Peer pressure
Tacit knowledge and skills
Action
Improved patient outcome
Patient concordance