ruling in or out a disease tests to rule out a disease you want very few false negatives high...

Ruling in or out a disease

Tests to rule out a disease

You want very few false negatives High sensitivity

Thus, if you get a negative test, it is likely a true negative

Mnemonic: SnOUT (High SeNsitivity rules OUT disease)

Example: D-dimer for DVT/PE

Test to rule in a disease

You want very few false positives High specificity

A positive test is likely to be a true positive

Mnemonic: SpIN (High Specificty rules IN disease)

Example: Pathology for malignancy

Prior and posterior disease probabilities

Prior and Posterior Probabilities

What you thought before + New Information = What you think now “What you thought before” = Prior (pre-test) probability

Disease prevalence Probability of disease given patient’s presentation

“New Information” = Test result “What you think now” = Posterior (post-test) probability

For positive dichotomous test, positive predictive value For negative dichotomous test, 1 – negative predictive value For multilevel and continuous tests, see chapter 4

Serial tests

2x2 table method for updating prior

probabilities• How to populate a 2 x 2 table• Example: Serological testing for TB

How to populate a 2 x 2 table

Given sensitivity, specificity, prevalence (prior probability)

Calculate positive predictive value, negative predictive value, etc.

Scenarios Applying test characteristics derived in one population to

another Applying test characteristics derived from a case / control

study Applying independent tests serially*

Assume sensitivity and specificity are intrinsic to test and independent of population

* Coming in chapter 8, “Multiple tests and multivariable decision rules”

How to populate a 2 x 2 table

Use prevalence to calculate D+ and D- totals

Use sensitivity and specificity to calculate A, B, C, and D

Use A, B, C, and D to calculate positive and negative predictive values

Example: Serological Test for TB

Anda-TB IgG ELISA test for anti-A60

antibodies

Sensitivity 76%

Specificity 92%

Prevalence Uganda: 30% SFGH: 5%

TB No-TB

Total

Positive

Negative

Total 300 700 1000 Have TB

1000 x 30% = 300

Don’t have TB 1000 – 300 = 700

Example: Serological Test for TB

Anda-TB IgG ELISA test for anti-A60

antibodies

Sensitivity 76%

Specificity 92%

Prevalence Uganda: 30% SFGH: 5%

TB No-TB

Total

Positive 228

Negative 72

Total 300 700 1000 Test positive if they have

TB 300 x 76% = 228

Test negative if they have TB 300 – 228 = 72

Example: Serological Test for TB

Anda-TB IgG ELISA test for anti-A60

antibodies

Sensitivity 76%

Specificity 92%

Prevalence Uganda: 30% SFGH: 5%

TB No-TB

Total

Positive 228 56

Negative 72 644

Total 300 700 1000 Test negative if healthy

700 x 92% = 644

Test positive if healthy 700 – 644 = 56

Example: Serological Test for TB

Anda-TB IgG ELISA test for anti-A60

antibodies

Sensitivity 76%

Specificity 92%

Prevalence Uganda: 30% SFGH: 5%

TB No-TB

Total

Positive 228 56 284

Negative 72 644 716

Total 300 700 1000 Positive Predictive Value

228/284 = 80.3%

Negative Predictive Value 644/716 = 89.9%

Positive Predictive Value 228/284 = 80.3%

Negative Predictive Value 644/716 = 89.9%

Example: Serological Test for TB

Uganda (prevalence 30%)

SFGH (prevalence 5%)

TB No-TB

Total

Positive 38 76 114

Negative 12 874 886

Total 50 950 1000

TB No-TB

Total

Positive 228 56 284

Negative 72 644 716

Total 300 700 1000 Positive Predictive Value

38/114 = 33.3%

Negative Predictive Value 12/886 = 98.6%

Sampling Scheme Matters

Cross sectional study Case control study

Positive Predictive Value 228/284 = 80.3%

Negative Predictive Value 644/716 = 89.9%

TB No-TB

Total

Positive 380 40 420

Negative 120 460 580

Total 500 500 1000

TB No-TB

Total

Positive 228 56 284

Negative 72 644 716

Total 300 700 1000 Positive Predictive Value

380/420 = 90.5%

Negative Predictive Value 460/580 = 79.3%

Odds and Probabilities

• Mmmm… pizza• Converting probabilities to odds and vice

versa

Mmmm… pizza

Imagine that you want to divide a pizza evenly between you and a friend

1 to 1 ratio of pizza between the two of you

Each gets 50% of the pizza

Mmmm… pizza

3 to 2 ratio of pepperonis between the two slices

One person gets 3/5 (60%) of the pepperonis

One person gets 2/5 (40%) of the pepperonis

Odds and Probabilities

Odds

‘Odds’ refers to the ratios of the two portions

Odds of 1 to 1 can be expressed as 1:1 or 1

Odds of 3 to 2 can be expressed as 3:2 or 1.5

Odds of 2 to 3 can be expressed as 2:3 or 0.67

Probabilities

‘Probabilities’ refers to the proportion of each portion to the whole

Probability of ½ is 0.5

Probability of 3/5 is 0.6

Probability of 2/5 is 0.4

Odds and Probabilities

Odds

Odds can range from 0 to ∞

To convert from probabilities: Odds = p / (1 – p)

Odds > probability

As p → 0 Odds ≈ probabilities

Probabilities

Probabilities can range from 0 to 1

To convert from odds: p = odds / (1 + odds)

Odds at low probabilities

Converting odds to Probability and vice versa

Odds of pizza is 0.33 (1:3) P = odds / (1 + odds) P = 0.33 / (1 + 0.33) P = 0.25 (¼)

Probability of pepperoni is 0.2 (⅕) Odds = p / (1 – p) Odds = 0.2 / (1 – 0.2) Odds = 0.25 (1:4)

Likelihood ratios for dichotomous tests

• Likelihood ratios• Example: Fetal fibronectin and preterm

labor

Likelihood ratios

Converts prior odds to posterior odds given a test result

Prior odds (A + C) / (B + D)

Posterior odds Positive test: A / B Negative test: C / D

Likelihood ratio is multiplier to go from prior odds to posterior odds

D+ D- Total

Positive A B A + B

Negative

C D A + C

Total A + C B + D NLR+ = sens/(1 – spec)

LR- = (1 – sens) / spec

Generalizing likelihood ratios

It is possible to have more than two results in a test Multilevel and continuous tests (chapter 4)

You can use likelihood ratios for these

LR+ and LR- don’t make sense

LRresult = P(result|D+) / P(result|D-)

LR method for updating prior probabilities

Step 1: Convert pretest probability to pretest odds

Step 2: Calculate appropriate likelihood ratio

Step 3: Multiply pretest odds by appropriate likelihood ratio to get post-test odds

Step 4: Convert post-test odds back to probability

Example: FFN for Preterm Labor

Fetal fibronectin: extracellular glycoprotein produced in the decidua and chorion Presence in vaginal secretions between 24 and 34

weeks gestation associated with preterm delivery ACOG recommends against screening asymptomatic

women “May be useful in patients at risk for preterm birth”

(within 7 days)

Example: FFN for Preterm Labor

Clinical scenario: 35 year old woman at 30 weeks gestation and history of preterm births, complains of “uterine tightening” Should she be given tocolytics and steroids for fetal

lung maturation

Fetal Fibronectin Sensitivity: 76% Specificity: 82% Prior probability of preterm birth within 7 days: 8% Sanchez-Ramos et al. Obstet Gynecol 2009;114:631–40

Example: FFN for Preterm Labor

Step 1: Convert prevalence to odds Odds = p/(1 – p)

0.08/(1 – 0.08) = .087

Remember when probability is small, odds ~ probability

Example: FFN for Preterm Labor

Step 2: Calculate likelihood ratios LR+ = sensitivity / (1 – specificity) LR- = (1 – sensitivity) / specificity

LR+ = 0.76 / (1 – 0.82) = 0.76 / 0.18 = 4.2

LR- = (1 – 0.76) / 0.82 = 0.24 / 0.82 = 0.29

Example: FFN for Preterm Labor

Step 3: Multiply pretest odds by appropriate likelihood ratio Post-test odds = pretest-odds x LR

For positive test: Post-test odds = 0.087 x 4.2 Post-test odds = 0.37

For negative test Post-test odds = 0.087 x 0.29 Post-test odds = 0.025

Example: FFN for Preterm Labor

Step 4: Convert post-test odds back to probability p = odds / (1 + odds)

For positive test: Post-test probability = 0.37 / (1 + 0.37) Post-test probability = 27%

For negative test Post-test probability = 0.025 / (1 + 0.025) Post-test probability = 2.48%

Example: FFN for Preterm Labor

Positive predictive value 61/227 = 26.9%

Negative predictive value 754/773 = 97.5%

preterm

not preterm

Total

Positive 61 166 227

Negative

19 754 773

Total 80 920 1000Sensitivity:

76%Specificity: 82%

Treatment and Testing Thresholds

• Quantifying costs and benefits• Testing thresholds for a perfect but risky

or expensive test

Quantifying costs and benefits

A “wrong” clinical decision carries cost Treatment of individuals without disease

Cost of therapy Discomfort, side effects, etc.

Failure to treat individuals with disease Pain and suffering Lost productivity Additional cases

Quantifying costs and benefits

Benefits of tests Increases probability of making the “right” decision Reduces costs from “wrong” decision

Costs of tests Cost of the test itself Discomfort and complications from performing the test May still lead to “wrong” decision (imperfect

sensitivity, specificity)

Quantifying costs and benefits

C = Cost of unnecessary treatment

B = Benefit forgone by failure to treat

T = Testing cost Cost of test itself, pain

and discomfort associated with procedure

Does not include cost of errors

Treatment Thresholds

Balance costs Cost of treatment x

probability of unnecessary treatment (no disease)

Net benefit forgone x probability of failure to offer treatment (probability of disease)

Cost of test

Treatment Thresholds

Treat: Probability of disease is high and/or Cost of treatment is low

Don’t treat: Probability of disease is low and/or Cost of failure to treat is low and/or Cost of unnecessary treatment is

high Cancer chemotherapy

Test: Test offers substantial

improvement in diagnosis and/or cost of test is low

Cost of treatment

Cost of treatment x probability of unnecessary treatment (no disease) C x (1 – P)

Cost of no treatment

Net benefit forgone x probability of failure to offer treatment (probability of disease) B x P

Lowest Cost Option

Black line is lowest cost option

Cost of treatment > Cost of no treatment Don’t treat (red)

Cost of treatment < Cost of no treatment Treat (green)

Cost of treatment = Cost of no treatment Treatment threshold (PTT)

Extending to dichotomous tests

Why you would do a test

Cost of treatment and/or failure to treat is high

Test will reduce misdiagnosis Test has good

performance characteristics

Test is used when diagnosis is ambiguous Pre-test probability is in

the middle

Why you wouldn’t do a test

The test is expensive

The test is unlikely to help The test’s performance

characteristics are limited You are already sure or

nearly sure of the diagnosis Pre-test probability is

very high or very low

Cost of a perfect test

If the test is perfect, the only cost to consider is the cost of the test T T is constant at all

probabilities

Lowest Cost Option

Black line is lowest cost option

Cost of no treatment < Cost of test Don’t treat (red)

Cost of treatment < Cost of test Treat (green)

Cost of test < Cost of empiric treatment/no treatment Test (yellow)

Lowest Cost Option

Treat/No Treat threshold P = C / (C + B)

Test/Treat threshold P = 1 – T/C

No Treat/Test threshold P = T/B

Example: EGFR in NSCLC

EGFR by PCR and fragment analysis for non-small cell lung cancer (NSCLC) Cost of test: $400 Assume sensitivity and specificity = 100% Prevalence of EGFR mutation (19% in males to 26% in females)

C = Cost of erlotinib $1300/mo x 3 months + risk of rash, diarrhea, occasional

interstitial pneumonitis ~ $4,000

B = Benefits of erlotinib in EGFR positive patients 3.2 months of progression free survival – cost of drug ~ $11,000

Example: EGFR in NSCLC

Imperfect tests

Must consider residual probability of being wrong

Lots of complex algebra (covered in the book)

Can also use Excel spreadsheet Available on course website

http://rds.epi-ucsf.org/ticr/syllabus/courses/4/2011/10/06/Lecture/tools/Treatment_Testing_Thresholds_Galanter.xls

Example

Summary

A dichotomous test is a test with two possible outcomes

We covered the definitions of sensitivity, specificity, prevalence, PPV, NPV, & accuracy

What you thought before + New information = What you think now

Probabilities can be updated using a 2 x 2 table

Odds are the ratio of two portions, Probabilities the proportion from the whole

Likelihood ratios convert prior odds to posterior odds given a test result

Treatment and testing thresholds allow you to estimate the lowest cost option