sensitivity, specificity and likelihood ratios

Sensitivity, Specificity and Likelihood Ratios

K.S. Chew

Faculty of Medicine and Health Sciences

Universiti Malaysia Sarawak

Email: [email protected]/25/2016 1

mailto:[email protected]

Sensitivity

• Proportion of patients with disease who are tested positive with a test

• A 100% sensitive test will not have any false negative results (although it may have a high rate of false positive results)

• Therefore, a negative result of a highly sensitive test means it is likely to be a true negative (it rules out the disease)

“SN-OUT”

1/25/2016 2

Specificity

• Proportion of patients without disease who are tested negative with a test

• A 100% specific test will not have false positive results (although it may have high rate of false negative results)

• Therefore, a positive result of a highly specific test means it is likely to be true positive (it rules in the disease)

“SP-IN”

1/25/2016 3

Sensitivity

Disease +ve Disease -ve

Test +ve a (TP) b (FP)

Test –ve c (FN) d (TN)

1/25/2016 4

TP = True positiveFN = False negativeFP = False positiveTN = True negative

Sensitivity = (a)/(a+c)

Positive Predictive Value




1/25/2016 5


Positive PV = (a)/(a+b)

Specificity




1/25/2016 6


Specificity = (d)/(b+d)

Negative Predictive Value




1/25/2016 7


Negative PV = (d)/(c+d)

Sensitivity and Specificity

Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html

1/25/2016 8

To increase sensitivity, shift to the left (purple line)

But by shifting to the left, it increases proportion of false positive, which means reduced specificity


Image taken from: http://library.med.utah.edu/WebPath/TUTORIAL/BIOSTATS/BIOSTATS.html

1/25/2016 9

To increase specificity, shift to the right (purple line)

But by shifting to the right, it increases proportion of false negative, which means reduced sensitivity

Example: Troponin assays

• First generation assay: cut-off 0.5 microgm/l

• 3rd generation assay: 0.05 – 0.10 microgm/l

• High-sensitive troponin (hsTn): 0.0030 microgm/l

• High-sensitive Roche Elecsys: 0.0014 microgm/l

• The diagnostic sensitivity of hsTn assays (ability to rule-out MI) are of the order of 90–95% when tested at the point of admission (still misses 5 - 10% of cases)

1/25/2016 10

Ref: Gamble et al, Br J Cardiol. 2013;20(4)

Causes of elevated troponins

• Myocardial ischemic conditions

• ACS

• Myocardial ischemic conditions other than ACS

• Systemic conditions

• Myocardial injury without ischemic insults

• Systemic conditions – renal failure, sepsis

• Specific identifiable precipitants – cardiac contusion, burns >30% BSA

1/25/2016 11


• A trade-off

• When sensitivity increases, specificity decreases

• When specificity increases, sensitivity decreases

1/25/2016 12

Image taken from: http://groups.csail.mit.edu/cb/struct2net/webserver/about.html

Receiver Operating Characteristics Curve

• When sensitivity increases, specificity decreases

• Therefore, when sensitivity increases, (1 – specificity) increases

• AUC – represents how good a test is

1/25/2016 13

Area under curve (AUC)

• Specificity is a measure of true negative; therefore (1 –specificity) is a measure of false positive

• While AUC of 1 represents a perfect test; AUC of 0.5 is a worthless test (a.k.a for every one true positive, there is an equal chance of getting one false positive)

• Interpretation:• 0.90 -1 = excellent • 0.80 - 0.90 = good• 0.70 - 0.80 = fair • 0.60 - 0.70 = poor• 0.50 - 0.60 = fail

1/25/2016 14

Example:

1/25/2016 15

Reichlin T, Hochholzer W, Bassetti S,

Steuer S, Stelzig C, Hartwiger S, et al.

Early Diagnosis of Myocardial Infarction

with Sensitive Cardiac Troponin Assays. N

Eng J Med 2009;361(9):858-67.

Methods

• Multi-center, n = 718, symptoms suggestive of MI

• Diagnostic accuracy of different troponin assays• Abbott–Architect Troponin I

• Roche High-Sensitive Troponin T

• Roche Troponin I, and Siemens Troponin I Ultra)

• vs standard assay (Roche Troponin T).

• Final diagnosis determined by 2 independent cardiologists: reviewing clinical history, physical findings, labs, ECG, echo, angio findings, etc

1/25/2016 16

Reichlin et al 2009

Results

1/25/2016 17

Reichlin et al 2009

Results

• AUC significantly higher for:

• Abbott–Architect Troponin I, 0.96 (95% CI 0.94 to 0.98)

• Roche High-Sensitive Troponin T, 0.96 (95% CI 0.94 to 0.98)

• Roche Troponin I, 0.95 (95% CI, 0.92 to 0.97)

• Siemens Troponin I Ultra 0.96 (95% CI, 0.94 to 0.98)

• standard assay, 0.90 (95% CI, 0.86 to 0.94)

1/25/2016 18

Reichlin et al 2009

Results

1/25/2016 19

Reichlin et al 2009

Likelihood Ratios

• Positive likelihood ratio refers to the likelihood of a patient with the disease to be tested as positive compared to a patient without the disease

• Negative likelihood ratio refers to the likelihood of patient with the disease to be tested negative as compared to a patient without the disease

• Every test has both LR (+) and LR (-)

1/25/2016 20

Likelihood Ratios

• LR are more helpful than sensitivity and specificity because sensitivity and specificity are derived from population where we already know whether they have or do not have the disease

• Whereas LRs tell us prospectively how a positive or negative test results affect the likelihood of patient to have a disease when we do not know whether they have it or not

• Likelihood ratios have factored in the sensitivity, specificity of the test (the TP, TN, FP, FN)

1/25/2016 21

Likelihood Ratios

• Positive likelihood ratio refers to the likelihood of a patient with the disease to be tested as positive compared to a patient without the disease

• LR (+)

• = (True positive)/(False positive)

• = (sensitivity)/(1-specificity)

• The higher LR (+), the better the test to RULE IN the disease

1/25/2016 22

Likelihood ratios

• Negative likelihood ratio refers to the likelihood of patient with the disease to be tested negative as compared to a patient without the disease

• LR (-) = (False Negative)/(True Negative)

• = (1 – sensitivity)/(specificity)

• The smaller the LR (-), the better the test TO RULE OUT the disease

1/25/2016 23

Usefulness of LRs

• To choose a diagnostic test

• E.g. which test would be the best to RULE IN a disease?

• Which test would be the best to RULE out a disease?

• To calculate a post-test probability (use Fagan Normogram)

1/25/2016 24

Example:

1/25/2016 25Collins et al, J Cardiac Failure 2015:21(1)

Fagan Nomogram

1/25/2016 26

Example:

• Why PERC score should only be used when Well’s criteria is in the low risk category?

• LR (-) of PERC is 0.17 (95% CI: 0.11 – 0.25)

• Ref: Carpenter CR, et al (2009). Differentiating low-risk and no-risk PE patients: the PERC score. J Emerg Med, 36 (3), 317-22

1/25/2016 27

PERC Score

• PERC score is a rule-out criteria for pulmonary embolism where if none of the 8 PERC criteria are present in a patient, PE can be ruled out clinically

• B = Blood in sputum (hemoptysis)

• R = Room air O2 Sat>95%

• E = estrogen or homonal use

• A = Age >50 years

• T = Thrombotic events (DVT, PE) or its possibility

• H = HR >/= 100/min

• S = surgery past 4 weeksl

1/25/2016 28

Determine your point of equipoise?

• Point of equipoise is the balance point when the risk-benefit of investigating further for PE vs risk-benefit of NOT investigating further for PE.

• Kline et al (2004) – point of equipoise for PE is 1.8%.

1/25/2016 29

Expolarating a LR (-) of 0.17

and a Post-test probability of 1.8%

Therefore, the pre-test probability

must be below 10%

1/25/2016 30

Determine that your post-test probability is no more than 1.8% (point of equipoise)

LR (-) for PERC

Wells criteria

Only in the low risk category

of Wells Criteria where the

probability of PE is below

10%. Therefore, PERC score

should be used only when the

Wells score is in the low risk

category

1/25/2016 31

Recommended Video Tutorials

• 6 short video series on sensitivity and specificity:

• https://www.youtube.com/watch?v=U4_3fditnWg&list=PL41ckbAGB5S2PavLIXUETzAmi5reIod23

• On likelihood ratios:

• https://www.youtube.com/watch?v=TzPvCSFZUSQ

1/25/2016 32

https://www.youtube.com/watch?v=U4_3fditnWg&list=PL41ckbAGB5S2PavLIXUETzAmi5reIod23

https://www.youtube.com/watch?v=TzPvCSFZUSQ

sensitivity, specificity and likelihood ratios

Health & Medicine