evidence-based practice in clinical psychology: what it is, why it matters, what you need to know...
TRANSCRIPT
Evidence-Based Practice in Clinical Psychology:
What It Is, Why It Matters, What You Need to Know
Bonnie Spring, Ph.D., ABPPBonnie Spring, Ph.D., ABPP
Northwestern UniversityNorthwestern University
Why it matters: EBBP Why it matters: EBBP RationaleRationale
improve quality and accountability for health care practice (IOM, 2001, Crossing the Quality Chasm)
shared vocabulary and concepts for transdisciplinary, biopsychosocial research, practice, health care policy
stimulate development of evidence base for behavioral treatments
Why it matters: Why it matters: Potentionally Useful InfrastructurePotentionally Useful Infrastructure
Clinical Practice Guidelines:Clinical Practice Guidelines: • Increasingly based on ongoing systematic review of research
(esp. RCTs) (e.g., USPTF, Cochrane, CDC/AHRQ)• Research reporting guidelines (CONSORT, TREND, QUOROM)• Evidence grading & knowledge synthesis systems (e.g.,
GRADE, AHRQ)• Policy, often coverage/reimbursement implications (VA/DOD,
CMS, NICE) (P4P?)
Evidence-Based Practice: Evidence-Based Practice: (life-long learning)• Question formulation, search strategies, critical appraisal• SUMSEARCH• Clinical Evidence, First Consult, BMJ updates, Best Evidence
Topics, CATCRAWLER, CATBANK – clinical scenario & bottom line
OverviewOverview
History of evidence-based practice History of evidence-based practice (EBP)(EBP)
Core elements of EBPCore elements of EBP EBP pedagogy in psychologyEBP pedagogy in psychology EBP pedagogy in other health EBP pedagogy in other health
disciplinesdisciplines Useful infrastructure and potential Useful infrastructure and potential
opportunities for synergyopportunities for synergy
Origins of Evidence-Based Practice
Emergence of Emergence of Evidence-Based Evidence-Based
MedicineMedicine1910 - Flexner report1910 - Flexner report : :155(31!)155(31!) 96 (1915) 76(1930) 96 (1915) 76(1930)
1972 - Archie Cochrane – epidemiology, health services research - 1972 - Archie Cochrane – epidemiology, health services research - Effectiveness and Efficiency: Random Reflections on Health Effectiveness and Efficiency: Random Reflections on Health ServicesServices
1973 – John Wennberg – widespread practice variation1973 – John Wennberg – widespread practice variation
1982 - clinical epidemiology determinants and consequences of 1982 - clinical epidemiology determinants and consequences of health care decisions (McMaster U – David Sackett, Gordon health care decisions (McMaster U – David Sackett, Gordon Guyatt)Guyatt)
1985 – IOM: 15% medical practices evidence-based [2001 1985 – IOM: 15% medical practices evidence-based [2001 Crossing Crossing the Quality Chasmthe Quality Chasm]]
1990 - Evidence-based medicine, Brian Haynes & Ann McKibbon – 1990 - Evidence-based medicine, Brian Haynes & Ann McKibbon – search strategiessearch strategies
1992-3 -Cochrane Collaboration1992-3 -Cochrane Collaboration
2000 - 2000 - Sackett - How to Practice and Teach EBMHow to Practice and Teach EBM
What do we mean by What do we mean by “evidence-based practice?”“evidence-based practice?”
Alternative Definitions of Evidence-Based Practice
Guidelines:Guidelines: (public health, medicine)public health, medicine) – – focus on problem/disorder & level of evidence for practices (based on systematic review) (e.g., NICE, VA, apa)
• ESTs:ESTs: (psychology)(psychology) focus on intervention (& disorder)
EBP:EBP: (psychology, medicine, nursing, social work) focus on decision-making about individual patients
Idiographic……….. Lifelong Learning
Nomothetic
APA Policy Statement APA Policy Statement adopted August 2005adopted August 2005
““Evidence-based practice in psychology is Evidence-based practice in psychology is the the integrationintegration of the best available of the best available researchresearch with with clinical expertiseclinical expertise in the in the context of context of patientpatient characteristics, culture, characteristics, culture, and and preferencespreferences.”.”
-adapted from IOM, 2001 & Sackett, 2000
Best available research evidence
Patient’s values, characteristics, and circumstances
Clinical Expertise
Clinical Decision-MakingClinical Decision-Making
Syllabus ProjectSyllabus Project
PromptPrompt: Does anyone on the list teach a : Does anyone on the list teach a course on evidence-based practice (EBP)? course on evidence-based practice (EBP)? Specifically, I am searching for syllabi that Specifically, I am searching for syllabi that cover one or more "legs" of the three-cover one or more "legs" of the three-legged EBP stool: a) research evidence, b) legged EBP stool: a) research evidence, b) clinical expertise, c) patient values, clinical expertise, c) patient values, preferences, characteristics. preferences, characteristics.
November, 2006
Listservs SampledListservs Sampled
ABCTABCT APA Division 12APA Division 12 SSCPNET (Section III, Div 12)SSCPNET (Section III, Div 12) CUDCPCUDCP APA Division 38APA Division 38 ABMRABMR SBM EBBM, MRBC, Obesity, CA SIGsSBM EBBM, MRBC, Obesity, CA SIGs
OutcomeOutcome
39 syllabi 39 syllabi 17 additional recommended articles 17 additional recommended articles
and booksand books 273 page document273 page document Discipline:Discipline:
30 psychology30 psychology 3 public health3 public health 3 medicine3 medicine 1 nursing 1 nursing 1 PE/health/sport studies1 PE/health/sport studies
140 requests140 requests
November, 2006
Evidence-Based PracticeEvidence-Based Practice
ModalModal
Course TitleCourse Title: CBT, EST, : CBT, EST, EVT, Psychological EVT, Psychological Interventions, Interventions, Psychotherapy ResearchPsychotherapy Research
TextsTexts:: Barlow, Barlow, Handbook Handbook Psychologic DisordersPsychologic Disorders, , Bergen & GarfieldBergen & Garfield Handbook of Handbook of Psychotherapy and Psychotherapy and Behavior ChangeBehavior Change
Content:Content: ESTsESTs
AdditionalAdditional
Additional TextsAdditional Texts::-Persons, -Persons, Case Case ConceptualizationConceptualization--Dawes, Dawes, House of CardsHouse of Cards
Additional ContentAdditional Content::-Assessment-Assessment-Case formulation, -Case formulation, functional analysisfunctional analysis-Clinical judgment-Clinical judgment-Diversity-Diversity-Iatrogenic effects-Iatrogenic effects-Research methods-Research methods
Content Distribution of Books on EBP
0
20
40
60
80
100
120
EBM (Sackettet.al)
EBN (Craig,Smyth, et.al)
SW (Gibbs) LIB (McKibbon) PSYCH(Norcross,
Beutler, Levant)
What is EBP?
How to do EBP
How to teach EBP
courtesy of Barbara Walker, Indiana University, 2006
Best available research evidence
Patient’s values, characteristics, and
circumstances
Clinical Expertise
Clinical Decision-MakingClinical Decision-Making
ResearcherDesign
Conduct Analysis
Reporting
SynthesizerLocateCritically appraiseMeta-analysis
ConsumerLocateAppraise quality & relevanceIntegrate
ClinicianCommunicateAssess patient
Deliver EBP
PatientUnderstandingPreferencesAccess
Researcher Training in Researcher Training in Psychology versus MedicinePsychology versus Medicine
PsychologyPsychology• DesignDesign
Correlational Correlational (convenience classes)(convenience classes)
Experimental (from Experimental (from animal studies)animal studies)
• ConductConduct Brief, tight controlBrief, tight control Little missing data; Little missing data;
replace casesreplace cases
• Analysis - Analysis - completercompleter• ReportingReporting
Clinical MedicineClinical Medicine• DesignDesign
Observational Observational (population)(population)
Clinical Trial –Clinical Trial –test of policy applied to population
• Conduct Long, intercurrent events Missing data;
• Analysis – ITT• Reporting – CONSORT
Researcher, Synthesizer, Researcher, Synthesizer, Consumer Training in AnalysisConsumer Training in Analysis
PsychologyPsychology• ANOVA/regressionANOVA/regression
Clinical MedicineClinical Medicine• Odds RatiosOdds Ratios
Epidemiology Terminology
Absolute risk (p[disease] in a particular population)Absolute risk (p[disease] in a particular population)
Relative risk (p[disease/exposed]/p[disease/unexposed)Relative risk (p[disease/exposed]/p[disease/unexposed)
Attributable risk (p[disease/exposed] -p[disease/unexposed)Attributable risk (p[disease/exposed] -p[disease/unexposed)
Number needed to harm (1/attributable risk)Number needed to harm (1/attributable risk)
Odds ratio Odds ratio (odds[disease/exposed]/odds[disease/unexposed])(odds[disease/exposed]/odds[disease/unexposed])
Clinical SignificanceClinical Significance
NNHNNH = 5. = 5. If 5 patients treated with If 5 patients treated with TX1, 1 would be more likely to have TX1, 1 would be more likely to have AE than if all had received TX0AE than if all had received TX0
NNTNNT = 13. = 13. 13 patients would need 13 patients would need to be treated with TX1 to see one to be treated with TX1 to see one success not seen with TX0success not seen with TX0
Reporting: Consort Flow Reporting: Consort Flow DiagramDiagram
Consolidated Consolidated Standards of Standards of Reporting Trials Reporting Trials (CONSORT)(CONSORT)
www.consort-statement.org
Excerpt from CONSORT checklistExcerpt from CONSORT checklist
METHODSParticipants
3 Eligibility criteria for participants and the settings and locations where the data were collected.
Interventions 4 Precise details of the interventions intended for each group and how and when they were actually administered.
Objectives 5 Specific objectives and hypotheses.
Outcomes 6 Clearly defined primary and secondary outcome measures and, when applicable, any methods used to enhance the quality of measurements (e.g., multiple observations, training of assessors).
Sample size 7 How sample size was determined and, when applicable, explanation of any interim analyses and stopping rules.
Randomization --Sequence generation
8 Method used to generate the random allocation sequence, including details of any restrictions
(e.g., blocking, stratification)
Randomization --Allocation
concealment
9 Method used to implement the random allocation sequence (e.g., numbered containers or central telephone), clarifying whether the sequence was concealed until interventions were assigned.
Randomization --Implementation
10 Who generated the allocation sequence, who enrolled participants, and who assigned participants to their groups.
Blinding (masking) 11 Whether or not participants, those administering the interventions, and those assessing the outcomes were blinded to group assignment.
When relevant, how the success of blinding was evaluated.
Evidence Synthesizer Evidence Synthesizer and Consumer Skillsand Consumer Skills
Best available research evidence
Patient’s values, characteristics, and
circumstances
Clinical Expertise
Clinical Decision-MakingClinical Decision-Making
ResearcherDesign
Conduct Analysis
Reporting
SynthesizerLocateCritically appraiseMeta-analysis
Evidence User LocateAppraise quality & relevanceIntegrate
ClinicianCommunicateAssess patient
Deliver EBP
PatientUnderstandingPreferencesAccess
SynthesizerSynthesizer: : Systematic Reviewer- Systematic Reviewer- explicit, systematic, transparent to avoid bias
Specific research question (PICO)Specific research question (PICO) Search protocol to select papers – key wordsSearch protocol to select papers – key words
systematic search of the literature (systematic search of the literature (EMBASE, CINAHLEMBASE, CINAHL, , Cochrane Controlled Trial register, DARE)Cochrane Controlled Trial register, DARE)
explicit inclusion and exclusion criteriaexplicit inclusion and exclusion criteria
Explicit, transparent rating of methodological Explicit, transparent rating of methodological qualityquality
Data extractionData extraction Analysis: qualitative or quantitativeAnalysis: qualitative or quantitative ConclusionConclusion Discussion of strengths and limitationsDiscussion of strengths and limitations
The 5 Step EBM Model for The 5 Step EBM Model for Evidence Users Evidence Users (Consumers) (Consumers)
Ask: formulate the question formulate the question Acquire:Acquire: evidence - search for evidence - search for
answersanswers Appraise: the evidence for the evidence for
quality and relevancequality and relevance Apply the results the results Assess the outcome the outcome
AskAsking: Well-Built Clinical ing: Well-Built Clinical QuestionsQuestions
BackgroundBackground:: What are effective What are effective treatments for bulimia nervosa?treatments for bulimia nervosa?
ForegroundForeground: : In patients with In patients with
PPatient: : binge eating disorderbinge eating disorderIntervention: does interpersonal
therapyComparison: compared to CBT reduceOutcome: frequency of binge episodes
Critically appraising the evidence
Use of standardized a priori appraisal Use of standardized a priori appraisal methods to answer:methods to answer:
Is the evidence Is the evidence validvalid??• Internal validity Internal validity
Is the evidence Is the evidence applicable/relevantapplicable/relevant??• External validityExternal validity
Is the evidence Is the evidence clinically significantclinically significant??
Clinical Decision-MakingClinical Decision-Making
Clinical epidemiology disciplineClinical epidemiology discipline• study of determinants and study of determinants and
consequences of clinical decisionsconsequences of clinical decisions• apply EBP/5A’s/critical appraisal at apply EBP/5A’s/critical appraisal at
clinical encounter to overcome clinical encounter to overcome automatic, unconscious decision-making automatic, unconscious decision-making biases biases (aka bad clinical intuition)(aka bad clinical intuition)
barriers between research and practicebarriers between research and practice
30 kg of guidelines per family doctor per year
25000 biomedical journals in print
8000 articles published per day
95% of studies cannot reliably guide clinical decisions
2001 Bazian Ltd
Clinical Decision-MakingClinical Decision-Making
Health InformaticsHealth Informatics disciplinediscipline
infrastructure, resources, devices, infrastructure, resources, devices, structures (e.g., algorithms, structures (e.g., algorithms, guidelines) needed to store, retrieve, guidelines) needed to store, retrieve, manage and use health information manage and use health information and the time and place that a and the time and place that a decision needs to be made. decision needs to be made.
-Decision support. -Decision support.
Secondary Synthesized Secondary Synthesized EvidenceEvidence
(AKA “evidence-based capitulation”)(AKA “evidence-based capitulation”) Research proliferates rapidly. Clinical Research proliferates rapidly. Clinical
performance demands increase. Practicing performance demands increase. Practicing clinicians too busy to use all EBM steps will all clinicians too busy to use all EBM steps will all patients. patients.
Increased focus on pithy clinical practice Increased focus on pithy clinical practice guidelines, synopses, and structured abstractsguidelines, synopses, and structured abstracts• MD ConsultMD Consult• ACP Journal ClubACP Journal Club• Cochrane Database of Systematic ReviewsCochrane Database of Systematic Reviews• ““Up-to-date”Up-to-date”• InfoPOEMS (Patient Oriented Evidence that Matters)InfoPOEMS (Patient Oriented Evidence that Matters)
Best available research evidence
Patient’s values, characteristics, and
circumstances
Clinical Expertise
Clinical Decision-MakingClinical Decision-Making
ResearcherDesign
Conduct Analysis
Reporting
SynthesizerLocateCritically appraiseMeta-analysis
ConsumerLocateAppraise quality & relevanceIntegrate
ClinicianCommunicateAssess patient
Deliver EBP
PatientUnderstandingPreferencesAccess
Clinically Supervised Training Clinically Supervised Training in Evidence-Based Treatmentin Evidence-Based Treatment
Needs work: 2005-2006 papers by Needs work: 2005-2006 papers by Woody and by Weissman Woody and by Weissman
Best available research evidence
Patient’s values, characteristics, and
circumstances
Clinical Expertise
Clinical Decision-MakingClinical Decision-Making
ResearcherDesign
Conduct Analysis
Reporting
SynthesizerLocateCritically appraiseMeta-analysis
ConsumerLocateAppraise quality & relevanceIntegrate
ClinicianCommunicateAssess patient
Deliver EBP
PatientUnderstandingPreferencesAccess
Patient PreferencesPatient Preferences Shared decision-making requires information only available to requires information only available to
patient (e.g., valuation of harms/hassles, alternative outcomes & patient (e.g., valuation of harms/hassles, alternative outcomes & treatments)treatments)
Utility assessment: All possible outcomes assigned a value between 0 : All possible outcomes assigned a value between 0 (death) and 1 (perfect health).(death) and 1 (perfect health).
Time trade-off approachTime trade-off approach• The proportion of life in a particular health state (e.g., severe The proportion of life in a particular health state (e.g., severe
depression) that you would give up to attain perfect health (e.g., 30%). depression) that you would give up to attain perfect health (e.g., 30%). Utility of that health state is 1-(30%) = .70Utility of that health state is 1-(30%) = .70
Standard gamble approachStandard gamble approach• The point where you are indifferent to the choice between spending the The point where you are indifferent to the choice between spending the
rest of your life in the health state in question and a gamble between rest of your life in the health state in question and a gamble between perfect health and instant death where the probability of perfect health perfect health and instant death where the probability of perfect health represents the utility of the health state.represents the utility of the health state.
Teaching evidence-based Teaching evidence-based practice = teaching a practice = teaching a
processprocess DidacticsDidactics Small groups, problem-based learning Small groups, problem-based learning Preceptorships/clinical supervisionPreceptorships/clinical supervision Standardized patients and evidence Standardized patients and evidence
stationsstations Embedded throughout curriculum Embedded throughout curriculum
Medical Decision Making in the NU-Medical Decision Making in the NU-FSM curriculumFSM curriculum
MDM-I (first week of medical school)MDM-I (first week of medical school)• Sensitivity, specificity, pre- and post-test probabilities, Sensitivity, specificity, pre- and post-test probabilities,
innumeracy, uncertainty in medicineinnumeracy, uncertainty in medicine MDM-II (last two weeks of M1 year)MDM-II (last two weeks of M1 year)
• EpidemiologyEpidemiology• StatisticsStatistics
MDM-III (beginning of M2 Spring Quarter)MDM-III (beginning of M2 Spring Quarter)• Decision analysisDecision analysis• Meta-analysisMeta-analysis• Cost-effectiveness analysisCost-effectiveness analysis• Clinical guidelinesClinical guidelines
M3 MDM (once a month in M3 year)M3 MDM (once a month in M3 year)• Review papers pertaining to clinical casesReview papers pertaining to clinical cases• Use of CATUse of CAT
Evidence-Based Behavioral Practice (EBBP)Evidence-Based Behavioral Practice (EBBP)
NIH Office of Behavioral and Social Sciences NIH Office of Behavioral and Social Sciences Research contract N01-LM-6-3512: Research contract N01-LM-6-3512:
Resources for Training in Evidence-Resources for Training in Evidence-
Based Behavioral Practice, Based Behavioral Practice,
2006 - 20112006 - 2011
OBSSR 5-Year PlanOBSSR 5-Year Plan Year 1Year 1: : developdevelop training websitetraining website, , Council, Council,
Scientific Advisory BoardScientific Advisory Board, , white paperwhite paper on training, on training, skills, competencies reflecting education in skills, competencies reflecting education in evidence-based behavioral practice (EBBP)evidence-based behavioral practice (EBBP)
Year 2Year 2: d: develop, implement a web-based, evelop, implement a web-based, research-focused training module(s)research-focused training module(s) on EBBP; on EBBP; field test in graduate curriculafield test in graduate curricula
Year 3Year 3: l: launch interactive web-based training aunch interactive web-based training courses; establish courses; establish practice networkpractice network, develop first , develop first EBBP EBBP clinical practice training moduleclinical practice training module
OBSSR 5-Year PlanOBSSR 5-Year Plan Year 4Year 4: With practice network, develop : With practice network, develop modulesmodules
on application of evidence-based on application of evidence-based clinical decision-clinical decision-makingmaking to intervention with specific cases to intervention with specific cases. . Field Field test in test in internship/residency/post-doctoral training internship/residency/post-doctoral training programs and practice network.programs and practice network.
Year 5Year 5: Link website to systematic reviews of : Link website to systematic reviews of behavioral interventions, treatment manuals, behavioral interventions, treatment manuals, outcome assessments. Develop and field test outcome assessments. Develop and field test clinical decision-making modules that clinical decision-making modules that integrate integrate patient preference and clinical competency patient preference and clinical competency assessmentsassessments..
SuggestionsSuggestions To enhance the evidence base for psychological To enhance the evidence base for psychological
treatments and support lifelong learning, clinical treatments and support lifelong learning, clinical psychology training might benefit from enhanced psychology training might benefit from enhanced coverage of: coverage of:
• Researcher skills in methods: clinical trial Researcher skills in methods: clinical trial design, analysis, reporting, synthesisdesign, analysis, reporting, synthesis
• Clinician training in 5-step (5A’s) EBP model – Clinician training in 5-step (5A’s) EBP model – cover 2 A’scover 2 A’s
Suggestions Suggestions • Psychology informatics could use infrastructure Psychology informatics could use infrastructure
development (PSYCinfo & Cochrane; development (PSYCinfo & Cochrane; library access; coverage in secondary synthesized ; coverage in secondary synthesized sources like Up-to-Date; practice-based research sources like Up-to-Date; practice-based research networks)networks)
• Psychology could use appropriate patient Psychology could use appropriate patient preference measures that support shared preference measures that support shared decision-makingdecision-making
• A discipline of clinical psychology decision-A discipline of clinical psychology decision-making needs to develop to systematize making needs to develop to systematize integration of research evidence, clinical integration of research evidence, clinical expertise, and patient clinical data and expertise, and patient clinical data and preferencespreferences
Concluding QuestionsConcluding Questions
What training modules and materials What training modules and materials would be helpful?would be helpful?
Will you partner with us to help Will you partner with us to help develop and try these out? develop and try these out?
The Evidence Pyramid forThe Evidence Pyramid forTreatment Effectiveness QuestionsTreatment Effectiveness Questions
***USE THE BEST EVIDENCE ***USE THE BEST EVIDENCE AVAILABLE***AVAILABLE***
Alternatives to evidence-based Alternatives to evidence-based medicinemedicine
Eminence based medicineEminence based medicine Eloquence based medicineEloquence based medicine Vehemence based medicineVehemence based medicine Nervousness based medicineNervousness based medicine
(Isaacs and Fitzgerald, 1999, BMJ)
Levels of Clinical Evidence in the Primary Levels of Clinical Evidence in the Primary Literature Literature (psycINFO, MEDLINE)(psycINFO, MEDLINE)
RCT, Practice Guideline RCT, Practice Guideline
Consensus Development Consensus Development ConferenceConference
RandomizedRandomizedControlled TrialControlled Trial
Quality Quality ImprovementImprovement
Cohort Studies, RiskCohort Studies, RiskCohort StudiesCohort StudiesEtiologyEtiology
Cohort Studies, Prognosis, Cohort Studies, Prognosis, Survival AnalysisSurvival Analysis
Cohort Studies, Cohort Studies, Case Control, Case Control, Case SeriesCase Series
PrognosisPrognosis
Randomized Controlled Randomized Controlled Trial, Double Blind, Clinical Trial, Double Blind, Clinical TrialsTrials
Double-Blind Double-Blind RandomizedRandomizedControlled TrialControlled Trial
TherapyTherapy
Search FiltersSearch FiltersMethodologyMethodologyType of Type of QuestionQuestion
EBM ResourcesEBM Resources
Pocket guides with web-linked Pocket guides with web-linked updates (Sackett; Guyatt & Rennie)updates (Sackett; Guyatt & Rennie)
Cochrane LibraryCochrane Library BMJ: BMJ: www.clinicalevidence.comwww.clinicalevidence.com Centre for EBM: Centre for EBM: http://http://
minerva.minervation.comminerva.minervation.com Centre for Evidence-based mental Centre for Evidence-based mental
health: health: www.cebmh.comwww.cebmh.com