Challenges for Evidence-based Diagnosis&

Course Review

Tom Newman

12/1/11

Challenges for EBD 01Dec11

Lecture Outline

Announcements Criticisms of EBM Challenges for EBD & rational decision

making Course review

Announcements We hope you read chapter 12. Optional reading on the website Take-home final will be posted after section

today – Due Thursday, 12/8/08 at 8:45 AM, no

exceptions– Your own work only, no collaboration– Feel free to look up cited articles for

clarification– We hope you enjoy it!

Next week: exam review and course evaluation

Historical background: changes that made EBM possible and necsssary

Increased ease of access to evidence via computers and the Internet

Development of science of clinical research

Multiple examples of interventions with strong basic science rationale that turned out to be harmful

More interventions offering only slight marginal benefit over alternatives; need to quantify and understand effect size

Criticisms of Evidence-Based Medicine -1

EBM over-values randomized blinded trials and denigrates other forms of evidence, including clinical experience.

– Historical background: EBM as a revolutionary questioning of authority

– Oversimplification of problems, "evidence hierarchy," reliance on checklists for critical apparisal

– No systematic reviews of parachutes*

*Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials BMJ  2003;327:1459-1461

Criticisms of Evidence-Based Medicine -2 Evidence-based treatment

recommendations tend towards the nihilistic

– May not recommend tests or treatments that some clinicians, professional societies or disease-specific advocacy groups believe are effective.

– Adoption of US Preventive Services Task Force evidence grading for other clinical situations

Criticisms of Evidence-Based Medicine -3

EBM has been or might be used by payers as an excuse to deny payment and limit clinician autonomy.

– Perceived need to recommend tests and treatments for fear they will not be reimbursed if optional

– Prostate cancer screening: Winners and Losers

– Sometimes real issue is probably costs, but e.g., Medicare cannot legally consider costs in coverage decisions

– What is the alternative to EBM?

Challenges for evidence-based diagnosis

Power of stories Difficulty estimating pre-test probabilities Difficulty finding applicable likelihood ratios Uncertainty about treatment thresholds

Clinicians are better at making decisions than at estimating any of the above

Stories and Statistics

People are more moved by individual stories than by statistics

"A single death is a tragedy; a million deaths is a statistic." *

Infant safety seats on airplanes– Requirement recommended by the AAP– FAA projected net increase in deaths due to

diversion to cars– TBN et al estimated if this did NOT occur, cost to

save 1 life ~$1.3 billion (@ $200/ticket)

*Joseph Stalin

Congressional Testimony

"The question, I think, Mr. Chairman, comes down to how many more children must die, how many more have to be hurt before we reach the threshold of FAA's ghoulish cost/benefit ratio?"

--Congressman Jim Lightfoot, Iowa

“Real” vs “Theoretical” Children

“The argument in support of the FAA’s resistance to the NTSB...is unreasonable on its face and ridiculous in its justification. It protects theoretical children driving in cars at the expense of real flesh-and blood infants whose safety is unquestionably compromised when flown as a lap-baby.”

Nader R, Smith WJ. Collision course: the truth about airline safety. Blue Ridge Summit, PA: TAB Books, 1994. Cited by Beshai D. Arch Ped Adol med 2003;157:953-4

Heuristics Used in Probability Estimation

Representativeness Availability Adjustment from an anchor

These heuristics can lead to biased estimates. See Chapter 12 for details.

Representativeness

If patient has typical features of a disease, we assign the disease high probability, even if prior probability was very low

Examples:– Chest pain radiating to the back aortic

dissection– “Worst headache of my life”

subarachnoid hemorrhage

Availability

The more easily you can imagine something happening, the higher the probability you assign it

Most recent and worst cases stand out Examples:

– Distorted view of risk of rare, serious illnesses from training in tertiary settings

– TN distorted view of risk of kernicterus (Consequences of making an error

factor into the probability estimation)

Anchoring People tend to estimate probabilities by

starting some place and revising probability up or down

This happens even if anchor is irrelevant Probability of pulmonary embolism study*

– “Do you think it is > or <1%?” OR “Do you think it is > or < 90%?”

– What do you think it is?

– Average estimate 23% vs 53%

Also used for estimating utilities

*Brewer et al. Med Decis Making 2007;27:203-11

Another bias

Example: I have a rule that allows me to say whether any sequence of 3 numbers is golden

The series 2, 4, 6 is golden Class suggest other series of 3

numbers and I’ll tell you if they are golden

See how quickly you can guess my rule.

* Mlodinow L. The Drunkard's Walk. Random House, 2008

Future of EBM More expensive tests and treatments Greater competition for resources (potentially

ugly) More guidelines, algorithms, computer-aided

decision making Need for leaders to understand how these

work, evaluate them critically, and help produce them!

Eventually need to acknowledge some tests or treatments cost too much

Need people who can explain these things to the public

Course Review (interactive)