diagnostic tests evaluation & uses
TRANSCRIPT
![Page 1: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/1.jpg)
Charles Charles CaraguelCaraguelCentre for Veterinary Epidemiological Research
Centre for Aquatic Health SciencesCentre for Aquatic Health SciencesAtlantic Veterinary CollegeAtlantic Veterinary College
University of Prince Edward IslandUniversity of Prince Edward Island
Diagnostic testsDiagnostic tests
evaluation & usesevaluation & uses
![Page 2: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/2.jpg)
Why is it so exciting?“Test performance is
critical to start a new life.” Me (2008)
1
![Page 3: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/3.jpg)
Basics (1): Variation & errors in estimation
SourcesSources• Sampling error (selection bias)
• Biological variation (time & space)
• Measurement error (diagnostics)
SemanticsSemantics
Health status (Binary outcome)
Test result (Binary outcome)
POSITIVE (T+)
NEGATIVE (T-)
DISEASED (D+)
NON-DISEASED (D-)
2
![Page 4: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/4.jpg)
Phase 1 - ”Laboratory“ validation
• Optimisation & standardisation of reagents
• Analytical sensitivity (ASe) & specificity (ASp)
Validation ObjectivesValidation Objectives““evaluation of a process to determine its evaluation of a process to determine its fitness for a particular usefitness for a particular use””
Dilution
3
![Page 5: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/5.jpg)
Phase2 - “Field” validation
• “Precision” (agreement sensus stricto)
Repeatability (overall intra-lab)
Reproducibility (overall inter-lab)
• “Accuracy” (performances)
Diagnostic Sensitivity (DSe)
Diagnostic Specificity (DSp)
Degree of mistakes made by the test
Phase 1 - ”Laboratory“ validation
• Optimisation & standardisation of reagents
• Analytical sensitivity (ASe) & specificity (ASp)
Validation ObjectivesValidation Objectives““evaluation of a process to determine its evaluation of a process to determine its fitness for a particular usefitness for a particular use””
4
![Page 6: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/6.jpg)
Basics (2): Basics (2):
Conditional ProbabilityConditional ProbabilityPr(Pr(AA//BB) = given ) = given BB, the probability of , the probability of AA
i.i. First set the primary information or condition (First set the primary information or condition (BB))
Health status (Health status (Se&SpSe&Sp); Test result (Predictive values)); Test result (Predictive values)
ii.ii. Then defined the component of interest (Then defined the component of interest (AA))
Health status (Predictive values); Test result (Health status (Predictive values); Test result (Se&SpSe&Sp))
5
![Page 7: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/7.jpg)
Tested positiveTested positive
Tested negativeTested negative
““ Non diseasedNon diseased ”” ““ DiseasedDiseased ””
Sp = Sp = Negative & Non dis. Negative & Non dis.
Non diseasedNon diseased
Se = Se = Positive & Diseased Positive & Diseased
DiseasedDiseased
““ True True PositivePositive ””
““ False False NegativeNegative ””
““ True True NegativeNegative ””
““ FalseFalsePositivePositive ””
6
![Page 8: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/8.jpg)
Accuracy DefinitionAccuracy DefinitionSe & SpSe & Sp
Diagnostic Sensitivity Diagnostic Sensitivity (true positive)(true positive)
Pr(Pr(TT++//DD++) =) =
Diagnostic SpecificityDiagnostic Specificity (true negative)(true negative)
Pr(Pr(TT--//DD--) =) =
the probability of the probability of TT++given given DD++, ,
the probability of the probability of TT--given given DD--, ,
7
![Page 9: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/9.jpg)
Legend of theLegend of the
2X2 TABLE2X2 TABLE
Sensitivity Sensitivity (true positive fraction)(true positive fraction)
Pr(Pr(TT++//DD++) = ) = a / (a + b)a / (a + b)
Specificity Specificity (true negative fraction)(true negative fraction)
Pr(Pr(TT-- //DD--) =) = d / (c + d)d / (c + d)
HealthStatus
Test Results
DD++
DD--
TT++ TT--
aa bb
cc dd
a + ca + c b + db + d
a + ba + b
c + dc + d
n = n = a+b+c+da+b+c+d
TP
FP
FN
TN
8
![Page 10: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/10.jpg)
Real applications in the Real applications in the ““fieldfield””
1. 1. Prevalence estimatePrevalence estimate
““farmers & veterinariansfarmers & veterinarians””
““ApparentApparent””
prevalence (prevalence (APrAPr))““TrueTrue””
prevalence (prevalence (TPrTPr))
Se / SpSe / Sp
Prevalence definitionPrevalence definition
APrAPr == TPrTPr = =
True prevalence formulaeTrue prevalence formulae
TPrTPr ==APrAPr + Sp + Sp -- 11
Se + Sp Se + Sp -- 11 True positive among D+
False positive among D-
Apparent prevalence formulaeApparent prevalence formulae
APrAPr = Pr Se + (1 = Pr Se + (1 -- TPrTPr) (1 ) (1 –– Sp)Sp)
# Positive# Positive
# Total (n)# Total (n)# Diseased# Diseased
# Total (n)# Total (n)
9
![Page 11: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/11.jpg)
Exercise (1):True prevalence estimation
During a surveillance screening, 18 cows tested positive
for Staphylococcus aureus out of 100 tested
(Test performances: Se = 72 % & Sp = 98 %)
Compute the following value:
Apparent Prevalence =
True Prevalence =
10
![Page 12: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/12.jpg)
Real applications in the Real applications in the ““fieldfield””
1. 1. Prevalence estimatePrevalence estimate
2. 2. Decision makingDecision making
““farmers & veterinariansfarmers & veterinarians””
““ApparentApparent””
prevalence (prevalence (APrAPr))
““TrueTrue””
prevalence (prevalence (TPrTPr))
Se / SpSe / Sp
Positive Predictive Value (PPV)Positive Predictive Value (PPV)
Given test is positive, what probability to be diseased?Given test is positive, what probability to be diseased?
Negative Predictive Value (NPV)Negative Predictive Value (NPV)
Given test is negative, what probability to be nonGiven test is negative, what probability to be non--diseased?diseased?
Primary Information: TEST RESULTS!Primary Information: TEST RESULTS!
(condition)(condition)
11
![Page 13: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/13.jpg)
NegNeg NegNeg NegNeg NegNeg NegNeg
NegNeg NegNeg NegNeg NegNeg NegNeg
NegNeg NegNeg NegNeg NegNeg NegNeg
NegNeg NegNeg NegNeg NegNeg NegNeg
NegNeg NegNeg NegNeg NegNeg NegNeg
PosPos PosPos PosPos PosPos PosPos
PosPos PosPos PosPos PosPos PosPos
PosPos PosPos PosPos PosPos PosPos
PosPos PosPos PosPos PosPos PosPos
PosPos PosPos PosPos PosPos PosPos
DiseasedDiseased
NonNon--diseaseddiseased
““ True True PositivePositive ””
““ False False PositivePositive ””
““ True True NegativeNegative ””
““ FalseFalseNegativeNegative ””
PPV = PPV = Positive & Diseased Positive & Diseased
PositivePositive
NPV = NPV = Negative & Non dis. Negative & Non dis.
NegativeNegative
NegativeNegative PositivePositive
12
![Page 14: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/14.jpg)
Positive Predictive value Positive Predictive value (PPV)(PPV) Strongly influenced by SpStrongly influenced by Sp
Pr(Pr(DD++//TT++) =) =
Negative Predictive value Negative Predictive value (NPV)(NPV) Strongly influenced by SeStrongly influenced by Se
Pr(Pr(DD--//TT--) =) =
the probability of the probability of DD++Given Given TT++, ,
the probability of the probability of DD--Given Given TT--, ,
Predictive Values DefinitionsPredictive Values Definitions
13
![Page 15: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/15.jpg)
Predictive Values FormulaePredictive Values Formulae
Positive Predictive ValuePositive Predictive ValuePr(Pr(DD++//TT++) = ) = a / (a + c)a / (a + c)
Negative Predictive ValueNegative Predictive ValuePr(Pr(DD-- //TT--) =) = d / (b + d)d / (b + d)
Test Results
DD++
DD--
TT++ TT--
aa bb
cc dd
a + ca + c b + db + d
a + ba + b
c + dc + d
n = n = a+b+c+da+b+c+d
TP
FP
FN
TN
14
![Page 16: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/16.jpg)
You test 100 cows for Staph. aureus infection. Sixty-five cows tested
negative by PCR and 35 tested positive. However, using culture as
a gold standard, 5/35 of the PCR-positives were disease-free and
14/97 of the seronegatives were diseased.
Fill the 2X2 table and compute AP, TP, Se, Sp, PPV, NPV:
Se =
Sp =
PPV =
NPV =
Exercise (2):2X2 table and more
TT++ TT--
DD++
DD--
15
![Page 17: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/17.jpg)
n*n*PrPr*Se*Sen*n*Pr(Pr(DD++&&TT++) )
n*n*Pr(Pr(DD--&&TT++) ) n*n*Pr(Pr(DD--&&TT++) )
n*n*Pr(Pr(DD++&&TT--) ) aa bb
cc dd
a + ca + c b + db + d n = n = a+b+c+da+b+c+d
Positive Predictive ValuePositive Predictive Value
Pr(Pr(DD++//TT++) =) =
Negative Predictive ValueNegative Predictive Value
Pr(Pr(DD-- //TT--) =) =
DD++
DD--
TT++ TT--
PrPr*(1*(1--Se)Se)
(1(1--Pr)Pr)*(1*(1--Sp)Sp) (1(1--Pr)Pr)*Sp*Sp
Pr(Pr(DD++) = ) = Prevalence(Prevalence(PrPr))
Pr(Pr(DD--) = ) = 1 1 -- PrPr
Pr(Pr(TT++//DD++) = Se) = SePr(Pr(TT-- //DD--) =) = SpSp
Bayesian Theorem: Bayesian Theorem: Pr(Pr(AA//BB) = ) = Pr(Pr(AA&&BB) / ) / Pr(Pr(BB)) Pr(Pr(AA&&BB)) = = Pr(Pr(BB) *) *Pr(Pr(AA//BB))
n take any value even 1n take any value even 1
PrPr*Se*Se
11
PrPr*Se*Se
PrPr*Se + *Se + (1(1--Pr)Pr)*(1*(1--Sp)Sp)
(1(1--Pr)Pr)*Sp*Sp
(1(1--Pr)Pr)*Sp*Sp + + PrPr*(1*(1--Se)Se)
When no a, b, c or d?When no a, b, c or d?but Se, Sp and Prevalencebut Se, Sp and Prevalence
16
![Page 18: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/18.jpg)
Real applications in the Real applications in the ““fieldfield””
1. 1. Prevalence estimate (risk analysis) Prevalence estimate (risk analysis)
2. 2. Decision makingDecision making
““farmers & veterinariansfarmers & veterinarians””
““ApparentApparent””
prevalence (prevalence (APrAPr))
““TrueTrue””
prevalence (prevalence (TPrTPr))
Se / SpSe / Sp
DSeDSe
DSpDSp
PrevalencePrevalence
Positive Predictive Value (PPV)Positive Predictive Value (PPV)
Given test is positive, what probability to be diseased?Given test is positive, what probability to be diseased?
Negative Predictive Value (NPV)Negative Predictive Value (NPV)
Given test is negative, what probability to be nonGiven test is negative, what probability to be non--diseased?diseased?
17
![Page 19: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/19.jpg)
in determining if she is infected with Staph. aureus. The PCR test
used assures 99 % accuracy (assumed 99% of DSe and 99% DSp).
The prevalence of cows infected with Staph. aureus and
presenting with clinical signs of mastitis of is 25%.
Define and compute the following values:
PPV =
NPV =
Exercise (3):Predictive value
Clarice is a 3 year-old cow showing
clinical signs of mastitis. The farmer
and the vet are interested
18
![Page 20: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/20.jpg)
Real applications in the Real applications in the ““fieldfield””
1. 1. Prevalence estimate (risk analysis) Prevalence estimate (risk analysis)
2. 2. Decision makingDecision making
3. 3. Surveillance ProgramSurveillance Program
““farmers & veterinariansfarmers & veterinarians””
““ApparentApparent””
prevalence (prevalence (APrAPr))
““TrueTrue””
prevalence (prevalence (TPrTPr))
Se / SpSe / Sp
Computation of sample size to declare a population Computation of sample size to declare a population ‘‘free of diseasefree of disease’’
DSeDSe
DSpDSp
PrevalencePrevalence
n / Sen / Se
Positive Predictive Value (PPV)Positive Predictive Value (PPV)
Given test is positive, what probability to be diseased?Given test is positive, what probability to be diseased?
Negative Predictive Value (NPV)Negative Predictive Value (NPV)
Given test is negative, what probability to be nonGiven test is negative, what probability to be non--diseased?diseased?
19
![Page 21: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/21.jpg)
QUESTIONS ?QUESTIONS ?
20
![Page 22: Diagnostic tests evaluation & uses](https://reader034.vdocument.in/reader034/viewer/2022042513/5882b78d1a28aba0198b9a94/html5/thumbnails/22.jpg)
21