a public lecture on the science & art of implementing evidence dave davis, md, ccfp, fcfp, frcpc...
TRANSCRIPT
A public lecture on the A public lecture on the Science & Art of Science & Art of
Implementing EvidenceImplementing EvidenceDave Davis, MD, CCFP, FCFP, FRCPC (hon) University of Toronto Faculty of Medicine
– Associate Dean, Continuing Education
– Principal Investigator, Knowledge Translation Program Ontario Guidelines Advisory Committee, Chair
If you don’t like If you don’t like thatthat title…title… Translating Guidelines into Practice Putting Guidelines in Place Using Evidence-based educational
principles to help clinicians put evidence into practice
Knowledge Translation:– old concept + new tools = better effect?
Vanessa Young, 1985-2000Vanessa Young, 1985-2000
Mild eating disorder (early satiety) diagnosed in 1998, in Oakville, Ontario
Seen by child psychiatrist and family doc., prescribed cisapride, with excellent results
1990; massive drug launch, all the bells & whistles 1992-98; subsequent, sporadic findings of cardiac
arrhythmias released by drug company bulletins, federal warnings (via print materials)
2000; Vanessa dies suddenly 2001; coroner’s inquest: family doctor especially
expresses inability to ‘keep up with the information overload’, like an ‘avalanche’
An outline; knowing when to nap
Definitions The Care Gap
– evidence for the gap in care – its extent & nature Causes of the gap
– problems with the learner, the message, the system
A Possible Solution – The creation and best use of guidelines– the question of translating knowledge into practice
“Knowledge translation is the effective and timely incorporation of evidence-based information into the practices of health professionals in such a way as to effect optimal health care outcomes and maximize the potential of the health system”
– Adapted from the Canadian Institutes for Health Research definition, 2001
Diffusion: distribution of information and the practitioners’ natural unaided adoption of policies and practices
Dissemination: communication of information to clinicians to improve their skills
Implementation: putting a guideline in place, involves effective communication, overcomes barriers by administrative and educational techniques
(after Lomas)...(after Lomas)...
What do CME & CPD mean?What do CME & CPD mean?’any & all ways by which physicians learn & change’ AMA
1972 courses
mailed materials,guidelines
peers,consultants
AV aids
patient ed
outreach visits
wwwsearch
reminders
Continuing professional Continuing professional developmentdevelopment
“…broader than ‘CME’, continuing professional development permits a consideration of many non-clinical topic areas, allows for a broader range of methods and settings. Further, it is more adult-learner centered….”
» Davis, Barnes, Fox, eds., The Continuing Professional Development of Physicians, AMA Press, 2003
Information overload
DEFINITIONSDEFINITIONS
Clinical practice guidelines are consensus and/or evidence-based statements of care intended to provide direction and assist decision-making in clinical care for both patients and clinicians..
• Adapted from the Institute of Medicine, 1990
Current practice
Ideal, evidence-based practice
clinical care gap
The clinical care gap
WARNING!!WARNING!!: this is the this is the interactive partinteractive part
1. Think about a gap in your practice, setting or experience
2. Define it
3. Figure out the why question – what are the barriers to full implementation of the guideline, evidence, whatever
4. (Figure out the ‘how to fix it’ question)
Exercise #1
Identify a clinical gap in practice/health care with which
you’re familiar
overuse
Acute pharyngitis– Fahey 1998
Acute Otitis Media– Delmar 1997
Acute bronchitis– 65-80% vs 20%– Gonzales 1997
?PSA screening ?Mammography for
low risk women age 40-50– Gotzsche, 2000
Others……(Ministry of Health, Ontario
data, 2001-04):
– Hysterectomies– Repeat C-Sections– Modified radical
mastectomy in breast CA
– Routine, pre-op chest X-rays, EKGs
– Lumbosacral X-rays for acute low back pain
– *Routine q6-12month echocardiograms in stable CHF
– *? Sleep studies
Underuse Pap Smears: Pirkis, 1998 CHF & ACE inhibitors
Hickling 2001– and beta blockers in the
elderly; McAlister 1999 Post MI patients
• Lipid lowering: Kong, 1998; Aronow, 1998
• ASA• Beta blockers
Atrial Fib & anticoagulation• Mendelson, 1999
Diagnosis of mental disorders• Craig and Boardman,
1997 ?Screening for colorectal cancer
>50
…and misuse– Beta blockers in
diabetics, asthmatics– Tricyclic
antidepressants in the presence of cardiac arrhythmias
– Cisapride (knowing what we know today)
Exercise # 2
Describe the causes of the gap
What causes the gap?What causes the gap? The evidence-to-practice puzzle
The educational deliverysystem
The clinicianThe evidence/guideline
Health CareSystem issues
•Patient•Team members
problems with the learner-clinician
age, experience, time (dis)incentives training
– emphasis on knowledge, not knowledge management
– inability to detect needs, evaluate performance
– self-directed learning– critical appraisal
type of practice competence motivation too narrow a definition
of ‘learner’ learning cycle:
awareness, agreement, adoption, adherence
No time…
No, Thursday’s out. How about never-is never good for you?
Dave’s top 10 reasons for not buying into CPGs 10) They change all the time 9) Guidelines, what guidelines? 8) I am too busy to adopt this
new stuff 7) Patient problems don’t fit
neatly into those little boxes 6) They were made in
Washington (Ottawa, Saskatchewan), wouldn’t apply here
Dave’s top 10, cont’d 5) I don’t trust all this EBM stuff 4) There were no family docs (left-
handed psychiatrists, etc) on the panel
3) MY patients expect ME to make decisions!
2) I already DO abide by the guidelines, yup, yessirree, 100%, all the time; that’s me - Mr. Guidelines.....and
1) MY patients are different!!
….problems with the guideline, evidence itself
– compatibility– complexity– cost– relative advantage– accessibility– format– patency of evidence, process of development– opportunity; trial-ability– Note the AGREE instrument
Producing & disseminating guidelines
1) selection of clinical question 2) literature searching 3) distillation/synopsis of literature 4) agreement by consensus, review 5) development of statement 6) endorsement of statement
7) distribution/dissemination7) distribution/dissemination
“Information management is like having your mouth to a firehose”
David Naylor, Dean, Faculty of Medicine, University of Toronto
2002
“It’s pretty simple, really: just review the world It’s pretty simple, really: just review the world literature every two weeks”literature every two weeks”
Sharon Straus, KT program,
University of TorontoLast week
…problems with the ‘Delivery System’ for CME
& CPD: does it work?
“Does CME work, Dave?
All these short courses - do they change how physicians practice?” Fraser Mustard
(not the guy in CLUE)
Dean, Faculty of Health Sciences, McMaster University
July 1977
Three Reviews of Three Reviews of ‘educational’ ‘educational’ interventionsinterventions
INCLUSION CRITERIA:– Randomized Controlled Trials– Replicable, educational interventions:
meetings, feedback, audiotapes, reminders, lectures, etc
– >50% practicing physicians/professionals– Objective outcomes of physician
performance or patient/health care status
StrategiesStrategies Educational materials Formal educational
meetings Outreach visits Local opinion leaders Patient mediated
strategies Audit/feedback
Reminders Mass media Combination
strategies
Other overall Other overall findings…findings… Needs Assessment (‘social
marketing’) appears to be important – the more the better (subjective needs, objective, gaps and barrier analysis)
No evidence much about long-term effects
Enabling materials – helpful?
Others’ findings (1999 –Others’ findings (1999 –2004):2004):[Cochrane Reviews- Thompson-O’Brien, Grimshaw, others] Most effects pretty much small to moderate at best,
INCLUDING– Multiple methods – Mailed materials
Reminders still mostly moderate-large effects, but few/no long-term studies
Methodology better understood, but studies often very messy, lack details, poorly designed
And… Quantitative methodology necessary but not sufficient
to understand change NO common theoretical base – mostly kitchen sink
research
Reason for the gap #4769: the ‘CME Process’
A Database of Physician Education & Change
www.cme.utoronto.ca/rdrbwww.cme.utoronto.ca/rdrb
The Research and Development Resource Base in CME
– educational, clinical & health services literature
– supported by the AMA, ACME, SACME,
Royal College of Physicians and Surgeons of Canada, the University of Toronto
Some comments on Some comments on
these reviews….these reviews…. CAUTIONS
– publication bias– screening bias– reporting gaps– very narrow,
quantitative, EBM-ish RCT bias
– focus of this review - change, not learning
– but.........
COMMENTS– size/scope/
nature of field– What we do
doesn’t work– What we don’t
do does
……..Exercise #3
What are the implications of this What are the implications of this gap, and its causes, for us as gap, and its causes, for us as
practitioners?practitioners?
What guidelines can’t do…
A CRITICAL LOOK AT A CRITICAL LOOK AT GUIDELINE GUIDELINE
DEVELOPMENTDEVELOPMENT
The AGREE The AGREE instrumentinstrument – –
The Not-all-guidelines-The Not-all-guidelines-are-equal Guidelineare-equal Guideline
SCOPE and PURPOSESCOPE and PURPOSE
RIGOR OF DEVELOPMENT
RIGOR OF DEVELOPMENT
EDITORIAL INDEPENDENCE
EDITORIAL INDEPENDENCE
CLARITY and PRESENTATION
CLARITY and PRESENTATION
APPLICABILITY
ST
AK
EH
OL
DE
R
INV
OL
VE
ME
NT
www.agreecollaboration.org
1. Scope and purpose
concerned with the overall aim of the guideline, the specific clinical questions and the target patient population.
Item 1. The overall objective(s) of the guideline is (are) specifically described
Item 2. The clinical question(s) covered by the guideline is(are) specifically described
Item 3. The patients to whom the guideline is meant to apply are specifically described
2. Stakeholder involvement
focuses on the extent to which the guideline represents the views of its intended users.
Item 4. The guideline development group includes individuals from all relevant professional groups
Item 5. The patients’ views and preferences have been sought
Item 6. The target users of the guideline are clearly defined.
Item 7. The guideline has been piloted among target users.
3. Rigor of development
relates to the process used to gather and synthesize the evidence, the methods to formulate the
recommendations and to update them.
Item 8. Systematic methods were used to search for evidence
Item 9. The criteria for selecting the evidence are clearly described
Item 10. The methods used for formulating the recommendations are clearly described
3. Rigor of development (continued)
Item 11. The health benefits, side effects, and risks have been considered in formulating the
recommendations
Item 12. There is an explicit link between the recommendations and the supporting evidence
Item 13. The guideline has been externally reviewed by experts prior to its publication
Item 14. A procedure for updating the guideline is provided
4. Clarity and presentation
deals with the language and format of the guideline.
Item 15. The recommendations are specific and unambiguous
Item 16. The different options for management of the condition are clearly presented
Item 17. The key recommendations are easily identifiable
Item 18. The guideline is supported with tools for application
5. Applicability
pertains to the likely organizational, behavioral and cost implications of applying the guideline.
Item 19. The potential organisational barriers in applying the recommendations have been discussed.
Item 20. The possible cost implications of applying the recommendations have been considered
Item 21. The guideline presents key review criteria for monitoring and/or audit purposes
6. Editorial independence
concerned with the independence of the recommendations and acknowledgement of
possible conflict of interest from the guideline development group.
Item 22. The guideline is editorially independent from the funding body
Item 23. Conflicts of interest of guideline development members have been recorded
Seems simple…So what’s all the fuss?
Current practice
Ideal, evidence-based practice
The clinical care gap: possible theory-to-The clinical care gap: possible theory-to-practice solutions, probable research practice solutions, probable research
questionsquestions
The informationInterventions The learner-target
knowledge translation strategies
1) ‘Solving’ the information problem – one example The Guidelines Advisory Committee,
Ontario – Joint body of the Ontario Medical Association and the Ministry of Health
and Long term Care, Ontario– Chooses a topic area; reviews all guidelines in that area; scores them
by the Cluzeau/AGREE instrument– Mounts them on a website
– Quick, 30 second synopsis– Parallel patient synopsis– Other links to QA tools, algorithms
– Simultaneous distribution/dissemination/implementation through medical schools, licensing body, professional associations, hospitals, etc
Other Efforts: Skolar, Cochrane, Ovid, Bandolier
2) Solving the ‘CME’/Intervention Problem – a 2) Solving the ‘CME’/Intervention Problem – a
possible modelpossible model BMJ 2003
Methods/ Stages
Awareness Agreement Adoption Adherence
Predisposing - Print material,- Lectures, - Academic detailing - Media campaigns
Enabling -Small groups,- Opinion leaders
- Pt. Education,- Opinion leaders
Reinforcing Reminders, Audit/ feedback
RemindersAudit/ feedback
NHS Consumer Health Information Web Site December 2001 5.2 million hits – 171,900 visitors (Powell & Clarke, 2002)
Fifty-eight per cent of GPs have been approached by patients with Internet healthcare information. Sixty-five per cent of the information presented by patients was new to GPs. (Wilson, 1999)
NOTE: communication skills
Solving the learner problem #3: Consumers can drive change, too: why not educate them?
A few final wordsA few final words– Large body of educational/change literature – largely unused
in guideline implementation – NO single effective change agent (except maybe
reminders); multiple methods may work best if they include the awareness-to-agreement continuum; methods work at different levels of change - predisposing, enabling & reinforcing
– Need to re-conceptualize ‘CME’, in order to incorporate models of ‘knowledge translation’, or guideline implementation; need to re-think targets
– Hope for the future: better models, more practical tools, information systems,Commonwealth initiatives in health, NICS, others, and
– Remember Vanessa
www.ktp.utoronto.ca
www.cme.utoronto.ca
www.gacguidelines.ca
For more For more information…….information…….