dichotomous tests thomas b. newman, md, mph september 27, 2012 thanks to josh galanter and michael...

Dichotomous Tests

Thomas B. Newman, MD, MPH

September 27, 2012

Thanks to Josh Galanter and Michael Shlipak

1

Overview Clarifications, chapter 1, chapter 2

material Definitions: sensitivity, specificity, prior

and posterior probability, predictive value, accuracy

2 x 2 table method Likelihood ratios - WOWO Probability and odds FP/FN confusion Test/treat thresholds 2

Clarifications

EBD errata on book website SLUBI= Self limited undiagnosed

benign illness – not a term I use with parents

3

Definitions: Sensitivity and Specificity

4

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

Sensitivity = A/ (A+C)

Specificity = D/ (B+D)

P.I.D. = Positive in Disease

N.I.H.= Negative in Health

Definitions: Positive and Negative Predictive value, Accuracy

5

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

PPV=A/(A+B)NPV=D/(C+D)

Accuracy = (A + D)/(A + B + C + D) = (A+D)/N Accuracy demonstration: screening for brain tumors

Definitions: Pretest (prior) and post-test (posterior) probability

6

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

Pretest probability =

ONLY IF SAMPLING IS “CROSS-SECTIONAL”!

(A+C)/(A+B+C+D)

Posttest probability = A/(A+B) or C/(C+D)

“Cross-sectional” sampling

7

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

PPV=A/(A+B)NPV=D/(C+D)

Subjects are sampled randomly or consecutively, so that the proportion with disease (pretest probability, prevalence) is clinically meaningful

“Case-control” sampling

8

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

PPV=A/(A+B)NPV=D/(C+D)

Subjects with and without disease are sampled separately

Proportion with disease is determined by investigator

Disease status

Has disease

No disease

Total

Test Result

Positive

A B A + B

Negative

C D C + D

Total A + C B + D A + B + C + D

Prevalence vs Pretest probability

Pretest probability is the more general term

For screening tests, pretest probability = prevalence

For diagnostic tests, pretest probability incorporates history and physical exam items

9

Post-test probability vs. Predictive value Posttest probability after a + test is the

same as positive predictive value Posttest probability after a – test is

1– negative predictive value

10

2 2 Table Method

Research vignette

“Tom, you need to call this mother. She’s really upset.”

11

Choroid Plexus Cyst

12

Fill in table

13

Pretest probability 0.0003 Sensitivity 33% Specificity 98.5%

Disease status

Trisomy 18

No Trisomy 18

Total

ChoroidPlexus Cyst

Present

Absent

Total

Likelihood Ratios

14

Likelihood ratios

A ratio of likelihoods:P(Result|Disease)P(Result|No Disease)

WOWO = With Over WithOut

Pretest odds x LR = Posttest odds (Prior odds x LR = Posterior odds)

15

What Tests Do•Their results change the probability of disease

Negative test Positive test

Reasurance TreatmentOrder a Test

•A good test moves us across action thresholds.

0% 100%

HIV+HIV-

16

Likelihood of Disease Depends on 2 Things

1. Where you started from (low, medium, high risk)

2. Length and direction of the “arrow”

Basic paradigm: – What we thought before test result

what we think now

17

Likelihood ratio Effect of test result

Very small (0.01) Greatly decreases P(disease)

Less than 1 (0.5) Decreases P(disease)

One No effect on P(disease)

More than 1 (2) Increases P(disease)

Very big (100) Greatly increases P(disease)

18

Likelihood Ratios Advantages

– Calculation of post-test probability easier (especially when disease is rare)

– Capture information for multi-level and continuous tests (next week)

Disadvantages– If either pretest or posttest probability is high

(~> 10%) you need to use odds (or a slide rule or calculator)

19

Switch to board

LR for the choroid plexus cyst example– Dichotomous test def of LR

Probability and odds

20

Can Use Slide Rule

21

False-negative confusion

Sensitivity of rapid strep test is 85% Therefore, false negative rate is 15% 15% is too high, so always culture to

confirm negative rapid strep tests

22

What’s wrong?Strep No Strep Total

Rapid Test + TP FP TP+FPRapid Test - FN TN TN+FN

TP+FN FP+TN

2 definitions of “false negative rate”– Def #1: 1-sensitivity = FN/(TP+FN). This one is easier because

it’s (assumed to be) constant.– Def #2: 1 - negative predictive value = FN/(FN+TN). This one is

harder because it depends on prior probability, but it is the one that should determine clinical decisions.

23

If prior probability of strep = 20% and specificity is 98%

False negative rate (def #2) = 15/407 = 3.7% NNC (number needed to culture) = 1/.037 = 27

to identify 1 false negative rapid test. (Pre-test probability of 20%)

At some prior probability of strep, culture after negative quick test is not indicated.

Strep No Strep TotalRapid test + 85 8 95Rapid test - 15 392 407Total 100 400 500

24

25

Sensitivity 85% Specificity 98% Prior probability = 20% Rapid test is NEGATIVE LR =

Try it with slide rule!

Similar examples:

Sensitivity of UA for UTI is only 80%, therefore always culture after a negative UA

Sensitivity of CT scan for subarachnoid hemorrhage is only 90%, therefore always do LP after a negative CT

False positive confusion is similar: 1-specificity vs. 1-positive predictive value

26

Test/Treat Thresholds

No test TreatTest

27

“X-Graph”

28

New “X-Graph”

29

Questions?

30