reading and evaluating the scientific and medical literature robert silbergleit, md department of...

Reading and evaluating the scientific and medical literature

Robert Silbergleit, MD

Department of Emergency Medicine

Goals

Reading• Philosophy

• Habit

• Strategies

Evaluating• Question

• Design

• Other Bias

• Statistics

Philosophy

The scientific literature is a discussion

Philosophy

Data are plentiful but truth is elusive

Habit

• You must read.

Habit

• You must read.

• There is too much to read.

“It’s all right, sweetie. In the information age, everybody feels stupid.”

Strategies

• Strategies for reading

– Grazing

– Searching

– Delivery

Strategies

• Information age strategic aids.– On-line journals– Weekly search agents

• Traditional strategic aids.– Review articles– Lectures– Journal clubs

Strategies

• Putting it together- a clinician’s strategy

– NEJM (or JAMA or BMJ)– Clinical specialty journal (or 2)– Special interest or subspecialty journal– Abstract service

Strategies

• A research scientist’s strategy

– Science (or Nature)– A research focus journal (or 2)– A weekly literature search agent

Strategies

• Pitfalls– Intimidation

– Education that is not education

– Honor and beware the opinion leaders

Strategies

• Strategies for skimming

– Table of contents– Abstract (don’t believe it)– Figures (or Results text)– Methods– Intro and discussion

Abstracts were found to contain errors or inaccurately represent the article 48% of the time, and ranged from 18% in one journal to 68% in another.

JAMA 1999;281:1110

Evaluating

• Assessing quality

• Being critical

• Determining value

Question

• Did they ask the right question?

• Did they answer the right question?

• Is the outcome they measured the one they are interested in?

Question

• What makes a good outcome measure?

– Meaningful– Accurate (Gold standard)– Objective– Distinct

Question

• Is the correct population studied, or the appropriate animal model used?

– Referral bias– Species differences

Design

• Randomized and controlled

• Observational (cohort, case-control)

Bias

In science we mean:

• Any systematic uncontrolled influence on data that favors a particular result.

Bias

But sometimes it really is the lay definition:

• A preference or inclination that inhibits impartial judgment.

Bias

• Publication bias

• Spectrum bias

Statistics

• Clinical v. statistical “significance”

• Hypothesis testing and the p-value

• Multiple comparisons

• Regression and confounding

Take home points

Science is a discussion

Make scientific reading a habit

Be skeptical, don’t be cynical