accuracy of poc resources to answer clinical questions ann mckibbon mls phd associate professor...

18
Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University [email protected]

Upload: khalil-spencer

Post on 29-Mar-2015

212 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Accuracy of PoC Resources to Answer Clinical Questions

Ann McKibbon MLS PhDAssociate ProfessorClinical Epidemiology & BiostatisticsMcMaster [email protected]

Page 2: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

POC Information Resources

Tremendous need for themAlready used heavily Traditionally thought of as being

– Positive– Neutral

Problems with content (itself or its application)Problems with use (system or user)

We need to address the challenges – Complexities – Their possible dark side

Page 3: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Information Needs in Practice

External information is vital in healthcare Numbers consistent over time, countries, disciplines, settings,

training levels

Clinic setting– Covell et al 1985 0.7 need/patient– Ramos et al 2003 0.8 need/patient

Hospital setting– Osheroff et al 1991 15/patient/day– Currie et al 2003 11/hour with EMRs

(range 1 – 25)

Page 4: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

• Clinicians use electronic resources during patient care time• 89% of them believe that electronic systems improve care• 75% of questions were answered

Page 5: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Problems with Information

Finding Time Recognizing needs Complexities

– Resources– Marketing/packaging

Systems– Move to simpler ones

Inability of resources to provide answers

Trust of producers & systems

Interpreting Time Recognizing needs Formulating questions Synthesizing evidence Coming up with the answer

– Matching evidence with clinical situation and patient

Application Trust of producers & systems

Page 6: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Seeking Information

Most use external resources– People/peers– Electronic information resources– Non electronic resources

Half of questions are pursued– 40% evidence found easily– 30% evidence found with difficulty– 30% evidence not found

Page 7: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Time to Find

MEDLINE Use Times per Search

43 minutes in Oregon 24 minutes in Iowa 23 minutes in Hamilton

Non MEDLINE times (e.g., PIER, UptoDate, CE, Google) range of 3 to 7 minutes

Sackett and Straus: On wards, resources will not be used if search time is > 30 seconds

Page 8: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Complexities of Systems

Size– MEDLINE 17,000,000

Version control– Cochrane reviews– Clinical practice guidelines

Multiplicity of resources– Numbers and kinds

Packaging of systems Data itself

Page 9: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Complexity of Systems

Clinicians often use systems poorly:

Scenario: Is PSA (prostate-specific antigen) a better biochemical predictor than alkaline phosphatase for detecting recurrent prostate cancer?

Approach: OVID MEDLINEprostate cancer (18K citations)reads for 5 minutesquitsprovides answer

Page 10: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Packaging—Cochrane Reviews

Whole content Cochrane Collaboration Database vendor (Wiley) PUBMED OVID

– MEDLINE– EBMR Cochrane Library– EBMR Full Text– EBMR All

InfoRetriever Google Scholar

Partial content ACP Journal Club Evidence Based Medicine Evidence Based Nursing InfoPOEMs OVID—EBMR ACP J Club Clinical Evidence Google Evidence-Based Guidelines …

Abstracts or full text? Free or pay?

Page 11: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Inability to Get Correct Answers

From Peers

Dutch study (Schaffsma) of 14 physicians Asked peers for answers to 12 questions 47% correct

Page 12: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Inability to Get Correct Answers

From Peers

Dutch study (Schaffsma) of 14 physicians Asked peers for answers to 12 questions 47% correct

– If evidence provided 83% correct– If evidence not available 35% correct

Page 13: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Inability to Get Correct Answers

From Electronic Information Resources

Medical, NP students: 45% right to 77% right

Australian GPs: 29% right to 50% right

McKibbon study: 39% right to 42% right

Page 14: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Inability to Get Correct Answers

From Electronic Information Resources

Medical, NP students: 45% to 77% 12% right wrong

Australian GPs: 29% to 50% 7% right wrong

McKibbon Study: 39% to 42% 11% right wrong

Page 15: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Providing Answers—EKGs

Electrocardiogram with computer interpretation

30 Internal Medicine residents (year 2 or 3)

Accuracy Without e-interpretation of data 48.9% With e-interpretation of data 55.4%

Page 16: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Providing Answers—EKGs

Electrocardiogram with computer interpretation30 Internal Medicine Residents (year 2 or 3)

Accuracy Without e-interpretation of data 48.9% With e-interpretation of data 55.4%

– If e-interpretration correct 68.1%– If e-interpretation wrong 48.3%

Page 17: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Trust

Producers of information trusted– WHO technical document

Trust of technology– Social conformity– Less vigilance – Diffusion of responsibility – Belief in infallibility

of technology

Page 18: Accuracy of PoC Resources to Answer Clinical Questions Ann McKibbon MLS PhD Associate Professor Clinical Epidemiology & Biostatistics McMaster University

Summary

Good systems exist although…

…systems that provide– right answer – to right question– at the right time– in the right format– with little or no effort

May not be here yet But we are getting closer…