Reading and evaluating the scientific and medical literature

Robert Silbergleit, MD

Department of Emergency Medicine

Goals

Reading• Philosophy

• Habit

• Strategies

Evaluating• Question

• Design

• Other Bias

• Statistics

Philosophy

The scientific literature is a discussion

Philosophy

Data are plentiful but truth is elusive

Habit

• You must read.

Habit

• You must read.

• There is too much to read.

“It’s all right, sweetie. In the information age, everybody feels stupid.”

Strategies

• Strategies for reading

– Grazing

– Searching

– Delivery

Strategies

• Information age strategic aids.– On-line journals– Weekly search agents

• Traditional strategic aids.– Review articles– Lectures– Journal clubs

Strategies

• Putting it together- a clinician’s strategy

– NEJM (or JAMA or BMJ)– Clinical specialty journal (or 2)– Special interest or subspecialty journal– Abstract service

Strategies

• A research scientist’s strategy

– Science (or Nature)– A research focus journal (or 2)– A weekly literature search agent

Strategies

• Pitfalls– Intimidation

– Education that is not education

– Honor and beware the opinion leaders

Strategies

• Strategies for skimming

– Table of contents– Abstract (don’t believe it)– Figures (or Results text)– Methods– Intro and discussion

Abstracts were found to contain errors or inaccurately represent the article 48% of the time, and ranged from 18% in one journal to 68% in another.

JAMA 1999;281:1110

Evaluating

• Assessing quality

• Being critical

• Determining value

Question

• Did they ask the right question?

• Did they answer the right question?

• Is the outcome they measured the one they are interested in?

Question

• What makes a good outcome measure?

– Meaningful– Accurate (Gold standard)– Objective– Distinct

Question

• Is the correct population studied, or the appropriate animal model used?

– Referral bias– Species differences

Design

• Randomized and controlled

• Observational (cohort, case-control)

Bias

In science we mean:

• Any systematic uncontrolled influence on data that favors a particular result.

Bias

But sometimes it really is the lay definition:

• A preference or inclination that inhibits impartial judgment.

Bias

• Publication bias

• Spectrum bias

Statistics

• Clinical v. statistical “significance”

• Hypothesis testing and the p-value

• Multiple comparisons

• Regression and confounding

Take home points

Science is a discussion

Make scientific reading a habit

Be skeptical, don’t be cynical