randy d ernst 1 , russell c hardie 2 , metin n gurcan 3 , aytekin oto 1 ,
DESCRIPTION
Randy D Ernst 1 , Russell C Hardie 2 , Metin N Gurcan 3 , Aytekin Oto 1 , Steve K Rogers 3 , Jeffrey W Hoffmeister 3 1. Department of Radiology, The University of Texas Medical Branch, Galveston TX 2. iCAD Inc. and University of Dayton, Dayton OH 3. iCAD Inc., Beavercreek OH. - PowerPoint PPT PresentationTRANSCRIPT
CAD Performance Analysis for CAD Performance Analysis for Pulmonary Nodule Detection: Pulmonary Nodule Detection:
Comparison of Thick- and Thin-Slice Comparison of Thick- and Thin-Slice Multi-detector CT ScansMulti-detector CT Scans
Randy D ErnstRandy D Ernst11, Russell C Hardie, Russell C Hardie22, , Metin N GurcanMetin N Gurcan33, Aytekin Oto, Aytekin Oto11,,
Steve K RogersSteve K Rogers33, Jeffrey W Hoffmeister, Jeffrey W Hoffmeister33
1. Department of Radiology, The University of Texas Medical 1. Department of Radiology, The University of Texas Medical Branch, Galveston TX Branch, Galveston TX
2. iCAD Inc. and University of Dayton, Dayton OH 2. iCAD Inc. and University of Dayton, Dayton OH 3. iCAD Inc., Beavercreek OH3. iCAD Inc., Beavercreek OH
PurposePurpose To compare the performance of To compare the performance of
a CAD (a CAD (QuickCue™QuickCue™, Beavercreek, , Beavercreek, OH) system in detecting lung OH) system in detecting lung nodules from nodules from thick- and thin-slicethick- and thin-slice multi-detector row CT.multi-detector row CT.
To evaluate the potential benefit To evaluate the potential benefit of CAD on radiologist sensitivity.of CAD on radiologist sensitivity.
Methods and MaterialsMethods and Materials 57 studies reviewed retrospectively 57 studies reviewed retrospectively Case selection:Case selection:
Obtained during a 5-month periodObtained during a 5-month period Referred from multiple departmentsReferred from multiple departments Contain at least 1 pulmonary nodule Contain at least 1 pulmonary nodule
but fewer than 10 nodules to localizebut fewer than 10 nodules to localize No significant miss-registration, No significant miss-registration,
breathing, surgical changes, pleural breathing, surgical changes, pleural effusions & atelectasiseffusions & atelectasis
Too many nodules to localizeToo many nodules to localize
Methods and MaterialsMethods and Materials
4-slice 4-slice multi-detector row CT multi-detector row CT (Lightspeed; GE Medical Systems)(Lightspeed; GE Medical Systems)
HQ setting with 7.5 mm/rotationHQ setting with 7.5 mm/rotation Standard-dose (160 - 270 mA, 120 Standard-dose (160 - 270 mA, 120
kVp)kVp) Images reconstructed at 5-mm Images reconstructed at 5-mm
(thick) and 2.5 mm (thin) slice (thick) and 2.5 mm (thin) slice thicknessesthicknesses
Methods and MaterialsMethods and Materials 140 nodules (3 mm - 25 mm) were 140 nodules (3 mm - 25 mm) were
identifiedidentified pre-CAD by radiologists pre-CAD by radiologists from thick-slice cases onlyfrom thick-slice cases only mean nodule size 7.3 mean nodule size 7.3 ±± 4.2 4.2 mm mm
Truth marks were mapped to the thick-Truth marks were mapped to the thick-slice 5mm data.slice 5mm data.
Gold standards for nodule truth created Gold standards for nodule truth created from post-CAD radiologist review from post-CAD radiologist review One gold standard for thick-sliceOne gold standard for thick-slice Separate gold standard for thin-sliceSeparate gold standard for thin-slice
CAD SystemCAD System (QuickCue™(QuickCue™, iCAD Inc., iCAD Inc.))
3D LungSegmentation
3D CandidateSegmentation
CalculateFeatures
DICOMImages
Classifier
DetectionMask
CAD SystemCAD System Candidates segmented by Candidates segmented by
thresholding and morphological thresholding and morphological processingprocessing
2D and 3D features computed for 2D and 3D features computed for each candidateeach candidate
Anatomical information (hilus, Anatomical information (hilus, airways, aorta, etc.) compared to airways, aorta, etc.) compared to reduce false positivesreduce false positives
A classifier applied for final decisionA classifier applied for final decision
CAD detected 72.1% (101/140) of the CAD detected 72.1% (101/140) of the thick gold standard truth nodulesthick gold standard truth nodules
CAD detected 35 additional CAD detected 35 additional radiologist-confirmed nodules, an radiologist-confirmed nodules, an increase of 25% (35/140) in sensitivityincrease of 25% (35/140) in sensitivity
5.6 (317/57) false-positives per case5.6 (317/57) false-positives per case 55 due to atelectasis55 due to atelectasis 18 due to scarring18 due to scarring
Review of Thick-Slice Review of Thick-Slice CAD ResultsCAD Results
Venn Diagram for ThickVenn Diagram for Thick
3
39 35
317
CADPre-CAD Review
Post-CAD ReviewGold Standard
101
0 0
CAD detected 80.7% (113/140) of CAD detected 80.7% (113/140) of the pre-CAD truth nodules.the pre-CAD truth nodules.
CAD detected 94 additional CAD detected 94 additional radiologist-confirmed nodules, an radiologist-confirmed nodules, an increase of 67.1% (94/140).increase of 67.1% (94/140).
4.6 (262/57) false-positives reported 4.6 (262/57) false-positives reported per case.per case. 70 due to atelectasis70 due to atelectasis 39 due to scarring39 due to scarring
Review of Thin-Slice Review of Thin-Slice CAD ResultsCAD Results
Venn Diagram for ThinVenn Diagram for Thin
0
26 94
262
CAD using thin-slicePre-CAD Review using thick-slice with detections mapped to thin-slice
Post-CAD Review of thin-sliceGold Standard
113
0 0
ComparisonComparisonThick-slice Thick-slice casescases
Thin-slice Thin-slice casescases
CAD CAD sensitivitysensitivity
72.1%72.1% 80.7%80.7%
Radiologist Radiologist sensitivity sensitivity increase increase after CADafter CAD
+25%+25% +67.1%+67.1%
FPsFPs 5.6 5.6 4.64.6
FROC Curve for CAD FROC Curve for CAD
0 2 4 6 8 10 12 140
0.2
0.4
0.6
0.8
1
Average False Positives Per Case
Prob
abili
ty o
f Det
ectio
n
Thin SliceThick Slice
CAD detections - Thick-SliceCAD detections - Thick-Slice
0 5 10 15 20 250
5
10
15
20
25
30
35
Size (mm)
Num
ber o
f Nod
ules
Radiologist DetectedRadiologist Missed
CAD detections -Thin-SliceCAD detections -Thin-Slice
0 5 10 15 20 250
10
20
30
40
50
Size (mm)
Num
ber o
f Nod
ules
Radiologist DetectedRadiologist Missed
5 primary lung cancers5 primary lung cancers 24 cases of metastatic cancer including 24 cases of metastatic cancer including
7 lymphomas, 4 breast, 4 head and neck, 2 7 lymphomas, 4 breast, 4 head and neck, 2 colon, 2 pancreas, 1 carcinoid, 1 seminoma, 1 colon, 2 pancreas, 1 carcinoid, 1 seminoma, 1 ovarian, 1 melanoma and 1 tracheal ovarian, 1 melanoma and 1 tracheal papillomatosispapillomatosis
23 cases of infection, including23 cases of infection, including 19 granulomatous disease either calcified, 19 granulomatous disease either calcified,
stable on follow-up or biopsy proven. 4 were stable on follow-up or biopsy proven. 4 were presumed infection that resolved with follow-up presumed infection that resolved with follow-up
1 case proved to be a thrombosed AVM1 case proved to be a thrombosed AVM 4 cases lost to follow up4 cases lost to follow up
Case Follow-upCase Follow-up
Example Example TPsTPs
Examples of Examples of nodules that are nodules that are detected by both detected by both radiologist and radiologist and CADCAD
Example TPsExample TPs Examples of nodules that are initially Examples of nodules that are initially
missed by radiologists then detected missed by radiologists then detected after reviewing CADafter reviewing CAD
Review of CAD ResultsReview of CAD Results Sources of false positivesSources of false positives
Vessel intersectionsVessel intersections Inaccurate lung segmentationInaccurate lung segmentation Partial volume effectsPartial volume effects Other lung abnormalities (scarring, Other lung abnormalities (scarring,
atelectasis)atelectasis)
Example FPsExample FPs
Review of CAD ResultsReview of CAD Results Sources of false negatives (missed Sources of false negatives (missed
nodules)nodules) Low density, irregularLow density, irregular Strong connectivity with vesselsStrong connectivity with vessels Imperfect candidate segmentationImperfect candidate segmentation Inaccurate lung segmentationInaccurate lung segmentation
5mm Thick slice
Example FNsExample FNs
5mm Thick slice
2.5mm Thin slice
5mm Thick slice
2.5mm Thin slice
ConclusionConclusionss Sensitivity and specificity of the CAD Sensitivity and specificity of the CAD
system increased when used with system increased when used with thin-slice scans versus thick-slice thin-slice scans versus thick-slice scans.scans.
CAD improved radiologist sensitivity CAD improved radiologist sensitivity on both thick- and thin-slice scans.on both thick- and thin-slice scans.
CAD improvement was greater for CAD improvement was greater for thin-slice scans.thin-slice scans.