ct image testing
DESCRIPTION
CT image testing. What is a CT image?. CT= computed tomography Examines a person in “slices” Creates density-based images. Three different window densities. Bone- densest only bone shows detail, the rest is too dark to make out - PowerPoint PPT PresentationTRANSCRIPT
CT image testingCT image testing
What is a CT image?What is a CT image?
CT= computed CT= computed tomographytomography
Examines a person Examines a person in “slices”in “slices”
Creates density-Creates density-based imagesbased images
Three different window densitiesThree different window densities
Bone- densest only bone shows Bone- densest only bone shows detail, the rest is too dark to make detail, the rest is too dark to make outout
Soft tissue- moderate density, bone Soft tissue- moderate density, bone is white, soft tissue is detailed, and is white, soft tissue is detailed, and “air space” is too dark“air space” is too dark
Lung- low density, bone and soft Lung- low density, bone and soft tissue white, and “air space” is tissue white, and “air space” is detaileddetailed
Bone WindowBone Window
Soft Tissue WindowSoft Tissue Window
Lung WindowLung Window
What a radiologist has to look atWhat a radiologist has to look at
How diagnoses are decidedHow diagnoses are decided
Look for “abnormalities” in densityLook for “abnormalities” in density
Why is this cumbersome?Why is this cumbersome?
Our project- dare to compareOur project- dare to compare
Test the new Test the new “unified LSU CT “unified LSU CT image”image”
HypothesisHypothesis
Unified LSU CT image:Unified LSU CT image: Equally accurateEqually accurate FasterFaster
MethodsMethods
Our testOur test ROC AnalysisROC Analysis ROCKit SoftwareROCKit Software
ROC ROC Receiver Operating CharacteristicsReceiver Operating CharacteristicsDeveloped in 1950’sDeveloped in 1950’sStatistical decision theory Statistical decision theory Used in business, economics, etcUsed in business, economics, etc
Goal: Is to be a meaningful tool for judging Goal: Is to be a meaningful tool for judging the performance of a diagnostic imaging the performance of a diagnostic imaging systemsystem
Example Example
X X Population Population
yy healthy healthy
z z diseased diseased Gaussian distribution Gaussian distribution Decision boundaryDecision boundary
Interested in optimum decision Interested in optimum decision boundary boundary
EXAMPLEEXAMPLENon-diseased
casesDiseased
cases
Threshold
EXAMPLEEXAMPLENon-diseased
casesDiseased
cases
more typically:
DECISION BOUNDARYDECISION BOUNDARY
TRUE NEGATIVETRUE NEGATIVE TRUE POSITIVETRUE POSITIVE FALSE NEGATIVEFALSE NEGATIVE FALSE POSITIVEFALSE POSITIVE What happens when you place the What happens when you place the
decision boundary?decision boundary?
DECISION BOUNDARYDECISION BOUNDARY
Threshold
Non-diseased
cases
Diseasedcases
FALSE NEGATIVE IN RED
TRUE POSITVE IN GREEN
DECISION DECISION BOUNDARYBOUNDARY
Threshold
Non-diseasedcases
Diseasedcases
FALSE POSITIVEIN WHITE
TRUENEGATIVE IN WHITE
SENSITIVITY TPFSENSITIVITY TPF
Threshold
Non-diseasedcases
Diseasedcases
SENSITIVITY = TP / TP + FNGREEN /GREEN+RED
FNF = 1-TPF
SPECIFICITY SPECIFICITY TNFTNF
Threshold
Non-diseasedcases
Diseasedcases
SPECIFICITY = TN / TN + FPLARGE WHITE / LARGEWHITE + SMALL WHITE
FPF = 1-TNF
PREVALENCEPREVALENCE
One of the important properties of One of the important properties of ROC is that it is prevalence ROC is that it is prevalence independentindependent
Low prevalence is rareLow prevalence is rare ExampleExample
ROC IS NOT PREVALENCE ROC IS NOT PREVALENCE DEPENDENT!DEPENDENT!
ROC ROC CURVECURVE
TP
F,
sens
itivi
tyFPF, 1-specificity
Entire ROC curve
COMPARING CURVESCOMPARING CURVES
ROC EXPERIMENTROC EXPERIMENT
DatasetsDatasets Two optionsTwo options
BINARYBINARY MULT-SCALE RATINGMULT-SCALE RATING
Fitting curvesFitting curves
RockitRockit Who developed Rockit?Who developed Rockit?
-Charles E. Metz @University of Chicago-Charles E. Metz @University of Chicago What is Rockit?What is Rockit?
-curve fitting software calculates-curve fitting software calculates points for a Roc curvepoints for a Roc curve
-calculates max probability of two -calculates max probability of two Gaussian Gaussian
distributionsdistributions What is a Roc Curve?What is a Roc Curve?
-describes how good the imaging systems is -describes how good the imaging systems is and accuracy of the given data and viewer’s and accuracy of the given data and viewer’s resultsresults
Example 1Example 1 100 patients, 40 negative, 60 positive100 patients, 40 negative, 60 positive
GivenGivenPP PP NN PP NN NN P…P…
Confidence RatingConfidence Rating1. Definitely Negative1. Definitely Negative2. Possibly Negative2. Possibly Negative3. Not sure3. Not sure
4. Possibly Positive4. Possibly Positive5. Definitely Positive5. Definitely Positive
This is the viewer’s results This is the viewer’s results 55 44 22 44 11 22 3….3….
Now, from the confidence rating we consider 1=N and 2, 3, 4, 5=P to interpret view’s results.Now, from the confidence rating we consider 1=N and 2, 3, 4, 5=P to interpret view’s results.Interpretation of viewer’s resultInterpretation of viewer’s resultPP PP PP PP NN PP P…P…
Comparing Comparing givengiven data and data and interpretationinterpretationTPTP TNTN FPFP FNFN 4545 25 25 0 0 13 13
(a) TP- disease is present, diagnosed correctly(a) TP- disease is present, diagnosed correctly(b) TN-disease is not present, diagnosed correctly(b) TN-disease is not present, diagnosed correctly(c) FP-person been diagnosed not having the disease, but the disease is present(c) FP-person been diagnosed not having the disease, but the disease is present(d) FN-person been diagnosed having the disease, but the disease is not present.(d) FN-person been diagnosed having the disease, but the disease is not present.
Now, we compute the FPF and TPF. In order to compute the FPF, you need this formula FPF=1-TP/TP+FN and TPF=1-TN/TN+FP.Now, we compute the FPF and TPF. In order to compute the FPF, you need this formula FPF=1-TP/TP+FN and TPF=1-TN/TN+FP.
Therefore, in our case FPF=1-45/45+13= .224 and TPF=1-25/25+0=0Therefore, in our case FPF=1-45/45+13= .224 and TPF=1-25/25+0=0
So, our first point is (.224, 0).So, our first point is (.224, 0).
Example 2Example 2 Now we can compute a second point on the Roc curve.Now we can compute a second point on the Roc curve.
We consider a different confidence rating. We consider a different confidence rating.
1, 2=N and 3, 4, 5=P1, 2=N and 3, 4, 5=PResults based on viewer’s answer from above Results based on viewer’s answer from above PP PP NN PP NN NN NN N….N….
The new Truth table is:The new Truth table is:TPTP TNTN FPFP FNFN 4646 28 28 0 0 15 15
Now the FPF=1-46 / 46+15 =.245Now the FPF=1-46 / 46+15 =.245 and TPF= 1-28/ 28+0 = 0 and TPF= 1-28/ 28+0 = 0
Now, we have our second point (.245, 0)Now, we have our second point (.245, 0)
Based upon the viewer’s results, a third and forth point can be created by changing the confidence Based upon the viewer’s results, a third and forth point can be created by changing the confidence ratings.ratings.1, 2, 3=N 1, 2, 3=N and and 4, 5=P4, 5=P ->third point->third point1, 2, 3, 4=N 1, 2, 3, 4=N and and 5=P5=P ->forth point->forth point
Therefore, five confidence ratings give you four points on the Roc curve.Therefore, five confidence ratings give you four points on the Roc curve.
Rockit inputRockit input For Condition 1: d For Condition 1: d
Enter the Total Number of Actually-Negative Cases (an integer):Enter the Total Number of Actually-Negative Cases (an integer):4040
Beginning with category 1 and separated by blanks,Beginning with category 1 and separated by blanks,Enter (on one line, integers only) the number of responses to Actually-Enter (on one line, integers only) the number of responses to Actually-Negative cases in each category:Negative cases in each category: 1 2 3 4 51 2 3 4 5 23 10 2 3 223 10 2 3 2
For Condition 1: d For Condition 1: d Enter the Total Number of Actually-Positive Cases (an integer):Enter the Total Number of Actually-Positive Cases (an integer):
6060 Beginning with category 1 and separated by blanks,Beginning with category 1 and separated by blanks, Enter (on one line, integers only) the number of responses to Actually-Enter (on one line, integers only) the number of responses to Actually- Positive cases in each category:Positive cases in each category: 1 2 3 4 51 2 3 4 5 2 2 3 10 432 2 3 10 43
RockitRockit
a, b parametersa, b parameters
- a represents - a represents separation of the separation of the Gaussian distributionsGaussian distributions
-b represents width of -b represents width of two Gaussian two Gaussian distributionsdistributions
Open excel template Open excel template to enter a, b to enter a, b parametersparameters
Roc curve is producedRoc curve is produced
Conventional Binormal ROC Curves
0
0.2
0.4
0.6
0.8
1
1.2
0 0.2 0.4 0.6 0.8 1 1.2
FPF
TP
F
a=2.10 b=0.90 Az=0.9445 Example