admin part 6: qi, ebm, and all that fun stuff! rebecca burton-macleod dr. sarah mcpherson r5, emerg...
TRANSCRIPT
![Page 1: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/1.jpg)
Admin Part 6: QI, EBM, and all that fun stuff!
Rebecca Burton-MacLeod
Dr. Sarah McPherson
R5, Emerg
Dec 6th, 2007
![Page 2: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/2.jpg)
Outline
• Evidence based medicine
• QI / QA
• Stats stuff
![Page 3: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/3.jpg)
Standards of evidence
• Levels of evidence– Class I: always
acceptable/safe/useful– Class IIa: acceptable/ safe,
results consistently positive– Class IIb: acceptable,
considered optional or alternative trmts
– Class III: unacceptable, may be harmful
– Intermediate: continuing research, no recommendation
• Strength of evidence– A: evidence-based
standards (RCT, cohorts, meta-analyses)
– B: guidelines (case-control, aggregate studies)
– C: options (x-sectional studies, case series, case reports, panel consensus)
![Page 4: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/4.jpg)
Impediments to decision making
• Patient related (content of history, completeness of history, unfamiliar terms, gender/ethnicity-bound, emotion/pain)
• Physician related (inadequate knowledge base, lack of data, other department distractions)
![Page 5: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/5.jpg)
Tools to assist decision making
• Guidelines
• Clinical pathways
• Computer based protocols
• Other evidence
![Page 6: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/6.jpg)
Guidelines
• “Systematically developed statements to assist practitioners and patient decisions about appropriate health care for specific clinical circumstances”
• For diagnostic or therapeutic interventions
![Page 7: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/7.jpg)
Guidelines
• Pros: Reduce variation in practice, decrease EP burden, promote proven interventions
• Cons: may be inadequate, provide contradictory information at times, may have goals other than patient benefit, lack of flexibility, may prevent EP from considering other possibilities, threat of use for legal purposes
![Page 8: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/8.jpg)
Development of guidelines
• Group to assess nature of evidence• Applicability of evidence to popn of interest• Fiscal implications of recommendations• Assess effect on health system• Technical and administrative support
required • Must have fiscal capabilities to develop (or
else adopt!)
![Page 9: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/9.jpg)
Clinical pathways
• “Translation of guidelines into locally developed functional tools that guide the process of care”
• Originated in industrial mgmt to reduce variation in production
• Aim to promote efficiency and reduce cost
• Constant monitoring and data evaluation
![Page 10: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/10.jpg)
Efficacy of clinical pathways
• Holmboe et al. Use of clinical pathways to improve the care of patients with AMI. Am J Med. 1999. 107:324.
• N=1122 pts (10 hosp with pathways, 22 without)
• Comparison of hospitals with critical pathways compared to those without
• No signif diff in length of hosp stay, mortality, use of proven medical therapies
• Did not look at physician compliance with pathways
![Page 11: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/11.jpg)
Strategies to change behaviour
• Remuneration
• Restrict resources
• Practice aids (chart reminders, different forms)
• Chart audit
![Page 12: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/12.jpg)
Computer-based protocols
• Require initiative to deviate from recommendation, as opposed to paper-guidelines which require initiative to move to recommended action
• May reduce potential sources of errors
![Page 13: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/13.jpg)
Improving compliance
• Burnstin et al. Benchmarking and QI: the Harvard ED Quality Study. Am J Med. 1999. 107:437.
• N=4876 medical records pre-intervention; 6005 post-intervention
• Reviewed all medical records for 5 teaching hospitals during 1mo period for 1 of 6 chief complaints and compliance with guidelines; pre and post-intervention
• Signif improvement with compliance with guidelines post-intervention
• 4% (signif) decrease in pt-reported problems with ED care
• The hospital with computer-based guidelines/medical records had greatest improvement in compliance
![Page 14: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/14.jpg)
Legal implications of guidelines
• Previously thought of as “recommendations”
• Implications of deviating from guidelines and potential legal effects are unknown
• Guidelines play a “relevant or pivitol role in the proof of negligence” in <7% of lawsuits in US
![Page 15: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/15.jpg)
Steps of EBM
• Definition of clinical question
• Collection of evidence
• Analysis of studies for validation, reliability, relevance
• Summary of most useful evidence
• Application to patient and critical appraisal
![Page 16: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/16.jpg)
The Sackett criteria…
• http://www.cche.net/usersguides/main.asp
• Centre for Health Evidence (on behalf of Evidence-Based Medicine Working Group)
• http://www.usersguides.org/
![Page 17: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/17.jpg)
Rosens keys to evaluating RCT• When reading a study, look to see that:
– pt popn is well defined and comparable to your own pt popn– info on eligible pts that were not enrolled– truly random treatment assignment and adequate blinding
procedures– impt confounders were measured and they did not differ
signif between groups studied– the % of pts lost to f/u and was a good systematic effort
made to perform good f/u– data categorizations that are clear and clinically useful– what agency funded research and what biases might have
been introduced
![Page 18: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/18.jpg)
Classic hypothesis testing
• Null hypothesis—no difference exists between two groups
• Alternative hypothesis—groups being compared are different
• Calculate p value—based on null hypothesis being true, probability of obtaining observation
• Accept/reject null hypothesis—if p<alpha value (usually 0.05) then null hypothesis rejected as false and alternative hypothesis accepted
![Page 19: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/19.jpg)
Quality improvement
![Page 20: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/20.jpg)
Plan
Do Study
Act
The model consists of two parts:
3 questions & a cycle for learning and improvement
What are we trying to accomplish?
How will we know that a change is an improvement?
What changes can we make that will result in improvement?
![Page 21: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/21.jpg)
Improvement Cycle“PDSA”
Plan:•State objectives•Make predictions•Make conditions explicit•Develop plan
Do:•Carry out the test•Document problems, observations.•Begin analysis
Study:•Complete analysis•Compare data to prediction•What did you learn?
Act:•Adopt, adapt or abandon?•Build knowledge sequentially
![Page 22: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/22.jpg)
HealthcareQualityImprovementModel
STEP 2:COLLECT BACKGROUND
DATA
STEP 1:FORM TEAMS
STEP 3:DEVELOP PROBLEM/ISSUE
STATEMENTS
STEP 4:DIAGRAM THE PROCESS
STEP 5:DETERMINE ROOT CAUSES OF
PROBLEMS/ISSUES
STEP 6:VERIFY CAUSES WITH DATA
STEP 7:CRITICAL REVIEW OF THE
LITERATURE
STEP 8:DEVELOP AND SELECT THE
SOLUTIONS
STEP 9:PLAN, TEST, IMPLEMENT THE
SOLUTIONS
STEP 10:MEASURE RESULTS
STEP 11:RE-EVALUATE AND
CONTINUOUS IMPROVEMENT
Getting Started
Getting Focused
Analysis
Decision Making
Implementation
Measurement
Continually Improve
“Trial and Learn” Plan - Do - Study - Act
measuring results and acting on themRe-evaluate and Continuous Improvement
“act, capture the gain and start all over”
DoPlan
StudyAct
![Page 23: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/23.jpg)
BreakthroughResults
Learning and improvem
ent
P DA S
P DA S
P DA S
P DA S
Theories, hunches,
& best practicesLearning and im
provement
P DA S
P DA S
P DA S
P DA S
Rapid and repeated use of the cycle helps
teams build knowledge sequentially
![Page 24: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/24.jpg)
QI Teams
• Formed for a specific purpose
• Specified life span
• Comprised of people who do / know the work
• Authority to make changes with approvals
![Page 25: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/25.jpg)
What about QA ?
• Quality assurance is the steps in place that hopefully help make a process safe
• Vs.
• Quality improvement which is looking at the system to see what improvements can be made
![Page 26: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/26.jpg)
How does QA function?
• Safety committee, M+M rounds, near miss/good catch forms…
• QI can lead to changes in QA
![Page 27: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/27.jpg)
Example
• QA process for radiology reports is currently all “discrepant” reports are sent to the duty doc for review (this includes all reports, even if normal, in which the doc did not enter an impression!)
• QI project could be developed to look at improving the process, so that only the reports with abnormal discrepant findings are sent…
• Any takers?
![Page 28: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/28.jpg)
Everyone’s favourite part…STATS!
![Page 29: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/29.jpg)
Define each of the following:
• Alpha value
• Type 1 error
• Type 2 error
• Power
![Page 30: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/30.jpg)
Define the following terms:
• Bias
• Pre-test probability
• Post-test probability
• Reliability
• Internal/external validity
![Page 31: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/31.jpg)
Write equations for each of the following:
• Power • Sensitivity• Specificity • Likelihood ratio (and what range is significant?)• Odds ratio• Relative risk • Negative predictive value• Positive predictive value• Number needed to treat
![Page 32: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/32.jpg)
Stats tests
• Differentiate between:– Students t test and Wilcoxon’s rank sum test
– One-way analysis of variance and Kruskal-Wallis test
– Chi-squared test and Fisher’s exact test
![Page 33: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/33.jpg)
Other important concepts (will make you look smart at journal
club)…
• Bonferonni correction
• Interim analysis of data
• Intention-to-treat analysis
• A priori probability
• Data dredging
![Page 34: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/34.jpg)
And now you will WOW Bryan…
![Page 35: Admin Part 6: QI, EBM, and all that fun stuff! Rebecca Burton-MacLeod Dr. Sarah McPherson R5, Emerg Dec 6 th, 2007](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de95503460f94ae45c8/html5/thumbnails/35.jpg)
Questions… ?(please direct all questions to Sarah…)