12/10/02harry bushar1 computerized thermal imaging breast cancer system 2100 (cti bcs2100)...
Post on 24-Dec-2015
214 Views
Preview:
TRANSCRIPT
12/10/02 Harry Bushar 1
Computerized Thermal Imaging Breast Cancer System 2100
(CTI BCS2100)Radiological Devices Advisory Panel
December 10, 2002Statistical PresentationHarry F. Bushar, PhD
12/10/02 Harry Bushar 2
Outline of Statistical Presentation
• Clinical Study Protocol• Objective• Design• Population• Demographics• Evaluation (Effectiveness + Safety)
• PMA Clinical Study• Effectiveness + Safety
• Amendment 4 Clinical Study• Effectiveness
• Amendment 5 Clinical Study• Effectiveness
• Amendment 7 Adjustment• Effectiveness
• Statistical Conclusions
12/10/02 Harry Bushar 3
Clinical Study Protocol
• Study Objective – “The objective of the study is to determine if
the CTI System, when used in conjunction with clinical examination and/or diagnostic mammography, increases the ability of physicians to differentiate benign from malignant, or suspicious, breast abnormalities.”
12/10/02 Harry Bushar 4
Clinical Study Protocol
• Study Design – prospective– blinded-to-histology– multi-center– intended to compare
• Level of Suspicion (LOS) (0-5) of malignancy of suspicious breast lesions for clinical examination or diagnostic mammography before BCS to
• LOS + (BCS Index of Suspicion (IOS) (0.00 – 100.00) of malignancy of suspicious breast lesions)
– biopsy as the “gold standard” for pathology.
12/10/02 Harry Bushar 5
Clinical Study Protocol
• Study Population– Original study population = 600 patients with
biopsy.– Actual study population = 2,407 patients with
biopsy.
12/10/02 Harry Bushar 6
Clinical Study Protocol• Study Demographics
– Gender• 2,364 Female• 15 Male• 28 Unknown
– Ethnicity• 53% Caucasian• 30% African American• 13% Latino• 2% Asian• 1% Other• 1% Unknown
– Age• 12% (< 40)• 55% (40 – 60)• 32% (> 60)• 1% Unknown
12/10/02 Harry Bushar 7
Clinical Study Protocol
• Primary Effectiveness on Overall Population– Evaluation
• Area Under the ROC* Curve (AUC) to compare results of diagnostic mammography (LOS) without BCS (IOS) and diagnostic mammography with BCS (LOS + IOS).
• Sensitivity• Specificity
– “The CTI System will be considered effective if its performance in conjunction with diagnostic mammography and/or clinical examination is clinically better than mammography and/or clinical examination alone.”
*ROC = Receiver Operating Characteristic
12/10/02 Harry Bushar 8
Clinical Study Protocol
• Secondary Effectiveness on Subpopulations– Mammographic Lesion Type
• Calcifications• Masses• Distortions
– Mammographic Lesion Size• < 0.5 cm• 0.5 – 1.0 cm• > 1.0 cm
– Mammographic Lesion Depth• As available
12/10/02 Harry Bushar 10
PMA Clinical Study Population
• The sponsor acquired BCS images from 2,407 patients at 6 US clinical sites from 12/20/96 through 4/30/01.
• The sponsor actually analyzed only those patients with both– mammography, not those with just clinical
examination, and – biopsy within 60 days.
12/10/02 Harry Bushar 11
PMA Training Clinical Study
• 700 patients, consisting of the first 220 patients + 480 patients randomly selected from among the next 1,912 patients, were used to set the following BCS IOS cut-off: ≥ 20.59 implies a recommendation for biopsy of
a given lesion and
< 20.59 implies a recommendation for short-interval follow-up of a given lesion.
12/10/02 Harry Bushar 12
PMA Testing Clinical Study
• 1,432 patients, enrolled from 12/20/96 through 10/30/00, were initially available to test the effectiveness of the BCS both in the original PMA and in Amendment 4.
• 769 patients, with 187 malignant lesions + 688 benign lesions, were actually included in the effectiveness evaluation both in the original PMA and in Amendment 4.
• Note that each patient had 1 to 4 lesions.• Note also that the sponsor assumes that lesions
within patient are independent.
12/10/02 Harry Bushar 13
PMA Clinical Study Results
• Primary Effectiveness by ROC AUC– The sponsor found, after excluding calcifications
alone, a statistically significantly greater ROC AUC for (IOS + LOS1) than for mammography LOS1 scores (1-4) alone at significance level = 0.05.
– The sponsor found, after again excluding calcifications alone, but expanding mammography LOS1 scores beyond (1-4) to include 2 additional intermediate categories (3.50 & 3.75), no statistically significant difference between ROC AUC for (IOS + LOS2) & (LOS2 alone) at significance level = 0.05.
12/10/02 Harry Bushar 14
ROC Plot of IOS+LOS1 Versus LOS1
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
1 - Specificity
Sens
itivi
ty IOS+LOS1
LOS1
12/10/02 Harry Bushar 15
ROC Plot of IOS+LOS2 Versus LOS2
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
1-Specificity
Sen
sitiv
ity
IOS+LOS2LOS2
12/10/02 Harry Bushar 16
PMA Clinical Study Results
• Safety– The following 4 adverse events occurred out
of 2,407 subjects (12/20/96 through 4/30/01):• 2 “mild”, possibly-related, resolved adverse events
were both associated with patient discomfort during positioning.
• 1 “serious” and 1 “mild”, not-likely-related, resolved adverse events were hospitalization for treatment of a pre-existing metabolic disorder and dizziness when sitting up after thermal imaging, respectively.
12/10/02 Harry Bushar 17
Amendment 4 Clinical Results
• Overall (187 malignant + 688 benign)– Sensitivity = 97.1% with
95% CI* = (94.1%, 98.8%)– Specificity = 14.3% with
95% CI = (12.1%, 16.6%)*CI = Confidence Interval
12/10/02 Harry Bushar 18
Amendment 4 Clinical Results
• Calcifications* (105 malignant + 368 benign)• Sensitivity = 94.8% with 95% CI = (89.6%, 97.8%)• Specificity = 9.2% with 95% CI = ( 6.9%, 12.1%)
• Masses (90 malignant + 322 benign)• Sensitivity = 100% with 95% CI = (96.7%, 100%) • Specificity = 18.0% with 95% CI = (14.6%, 21.9%)
• Distortions (16 malignant + 15 benign)• Sensitivity = 100% with 95% CI = (82.9%, 100%)• Specificity = 16.7% with 95% CI = ( 3.9%, 40.2%)
*The sponsor interpreted these effectiveness clinical results by lesion type to specifically exclude calcifications alone.
12/10/02 Harry Bushar 19
Amendment 4 Interpretation
• The sponsor’s initial rejection of overall effectiveness, followed by the sponsor’s differential findings among the 3 lesion type sub-populations clearly indicates exploration: – which does require confirmation, – which must be based on new data.
12/10/02 Harry Bushar 20
PPMA* Amendment 5 Population
• Study population = 275 additional patients.– Gender
• 274 female• 1 male
– Ethnicity• 65% Caucasian• 31% African American• 3% Latino• 1% Asian or Unknown
– Age• 13% (< 40)• 60% (40 – 60)• 27% (> 60)
*PPMA = Post-PMA
12/10/02 Harry Bushar 21
PPMA Amendment 5 Study
• 275 additional patients, enrolled from 11/1/00 through 4/30/01 at 3 of the 6 original US clinical sites, were initially available for confirmation of the effectiveness of the BCS in Amendment 5.
• 173 patients, with 43 malignant + 159 benign lesions, were actually included in the Amendment 5 effectiveness evaluation.
• Note that each patient had 1 to 3 lesions. • Note also that the sponsor assumes that lesions
within patient are independent.
12/10/02 Harry Bushar 22
PPMA Amendment 5 Results
• Overall (43 malignant + 159 benign)– Sensitivity = 93.8% with
95% CI = (84.0%, 98.5%)– Specificity = 20.0% with
95% CI = (14.9%, 26.0%)
12/10/02 Harry Bushar 23
PPMA Amendment 5 Results
• Calcifications (24 malignant + 87 benign)• Sensitivity = 93.1% with 95% CI = (77.9%, 99.0%)• Specificity = 17.4% with 95% CI = (11.1%, 25.5%)
• Masses* (15 malignant + 63 benign)• Sensitivity = 93.3% with 95% CI = (72.1%, 99.7%)• Specificity = 25.4% with 95% CI = (16.6%, 36.0%)
• Distortions (8 malignant + 4 benign)• Sensitivity = 75.0% with 95% CI = (40.0%, 95.4%)• Specificity = 25.0% with 95% CI = ( 1.3%, 75.1%)
*The sponsor interpreted these effectiveness results by lesion type to specifically include only masses.
12/10/02 Harry Bushar 24
Amendment 7 Adjustment• The sponsor’s attempted Bonferroni adjustment, in response to FDA
deficiency 1.a), by – widening sensitivity/specificity CI estimates, which are – based on a simple direct combination of exploratory + confirmatory
clinical data (PMA/Amendment 4 + PPMA Amendment 5), to– test a theoretically possible hypothetical set of hypotheses for
• 7 (lesion types) or • 63 (lesion types, sizes, & depths),
– which hypotheses are not explicitly included in the protocol, • Is not statistically acceptable, because
– the sponsor simply estimates sensitivity/specificity CI’s for various sub-populations,
– without actually statistically testing any hypotheses.• Therefore, there are no multiple comparisons requiring adjustment.
12/10/02 Harry Bushar 25
Statistical Conclusions• Diagnostic mammography, not just clinical examination, is required
for use of BCS.• The sponsor’s primary effectiveness demonstration, using ROC
AUC, loses statistical significance, when mammography LOS (1-4) is expanded by just 2 additional intermediate categories (3.50 & 3.75), after excluding calcifications alone.
• The sponsor’s initial rejection of overall sensitivity, followed by rejection of calcification-alone sensitivity, indicates exploration:– which requires confirmation, – which requires new data.
• The sponsor’s attempted Bonferroni adjustment, by widening sensitivity/specificity CI estimates, is not statistically acceptable.
top related