“one certainly cannot predict future events exactly if one cannot even measure the present state of the universe precisely!”

Stephen Hawking, A Brief History of Time

Redefining gold standard diagnosis and application for risk factor analysis in cervical dysplasiaEmily King, MS4 & MPH candidateOregon Health & Science University

Epidemiology

MedicinePathology

Biostatistics

• Cohen’s kappa statistic• Sensitivity, specificity & predictive value

Misclassification biasRisk estimation

Immunohistochemistry

Cervical dysplasia

Medicine & Pathology

Interesting and relevant?

Screening programs using Pap smears and colposcopic-biopsy have reducedthe incidence of cervical cancer worldwide.

Human papillomavirus is prevalent in >90% of cervical cancers.

The natural history of cervical pre-cancer (dysplasia) is unclear.• Screening and treatment• HPV biology is incompletely understood• Limitations of methods of HPV detection • Women become exposed to HPV at a time when the cervical epithelium

may be particularly susceptible

The use of immunohistochemistry is prevalent.

Immunohistochemical Markers

Mitosis

Synthesis

p16

pRb HPV E7

G2 G1

Synthesis

G0 p16

pRb HPV E7

Degradation

Mitosis

HR-HPV infection leads to enhanced expression of the p16 gene product.

Current understanding is that CIN 2 and CIN 3 are most likely the result ofpersistent HR-HPV infections whereas CIN 1 may be the result of LR-HPV infection.

This is consistent with observations that CIN 2 and CIN 3 stain for p16 in a different pattern than CIN 1.

Why should you care?

…because we are trying to identify meaningful biologic or social entities aboutwhich we can advise patients so that risk can be minimized or prognosis fullyunderstood.

…because management is determined by pathologic diagnosis & guidelines for adolescents (2007): observation of CIN 2. GOAL is to limit overtreatment and treat legitimate disease appropriately.

TAKE HOME: Our (pathology, radiology, clinical medicine) inabilityto precisely and accurately classify outcome is affecting theconclusions we draw from observational studies and subsequentlythe advice or management we recommend for patients.

= OBSERVATIONAL STUDIES

Epidemiology

1. Previous estimates of risk are biased toward the null (OR = 1.0) because of misclassification bias.

2. This is evidenced by non-significant ORs in previous studies andrelatively poor diagnostic methods (compared to diagnosis withH&E + p16).

3. Using IHC (p16) can improve precision and accuracy and thereforeminimize misclassification bias.

4. This will allow calculation of more valid ORs, which will moreclosely estimate true risk in the population.

Tell ‘em what your gonna tell ‘em:

10.1 0.5 5 10Odds Ratio

Risk Factors for CIN2+Based on old H&E “gold standard diagnosis”

v Smoking (1.2-1.7) (McIntyre, 2005)

Low Education (1.2-2.4) (Khan, 2005)Black Race (0.4-0.9) (Khan, 2005)

Parity (0.8-1.07) (Belinson, 2008)

Contraceptives (0.9-1.1) (Castle, 2005)

Age (0.8-2.2) (Belinson, 2008)

HPV titre (2.5-65.0) (Song, 2006)

10.1 0.5 5 10Odds Ratio

Risk Factors for CIN2+Based on old H&E “gold standard diagnosis”

v Smoking (1.2-1.7) (McIntyre, 2005)

Low Education (1.2-2.4) (Khan, 2005)Black Race (0.4-0.9) (Khan, 2005)

Parity (0.8-1.07) (Belinson, 2008)

Contraceptives (0.9-1.1) (Castle, 2005)

Age (0.8-2.2) (Belinson, 2008)

HPV titre (2.5-65.0) (Song, 2006)

Biostatistics

Misclassification bias

Outcome

Expo

sure +

-

+ -

a b

c d

GOAL: minimize risk, understand prognosisValid observational study design requires minimization of sources of bias b/c bias willmay have effects on the conclusions of the study.

Outcome

Expo

sure

+

-

+ -

95 50

5 50Ex

posu

re

+

-

+ -

77 68

23 32

OR 19

n=200

38

20

202

OR 1.57

Biostatistics

Truth

What we can actually measure

Hypothesis:

1. Assuming that a diagnostic method that is more precise and more accurate leads to reduction in sources of bias (ie. misclassification), the odds ratio calculated using this method to determine outcome will be more valid than otherdiagnostic methods.

2. Therefore, this OR will most closely estimate the true risk in the population.

3. Assuming non-differential misclassification bias has been present in previous studiesand ORs from these studies are biased toward the null, then using a new diagnosticmethod which introduces less bias will result in ORs more extreme than previous estimates.

Specific Aims

1. Utilize two IHC markers to improve precision/accuracy inclassification of grades of CIN.

-Kappa, sensitivity/specificity, predictive value

2. Calculate odds of having an outcome (CIN2+) – as defined by different diagnostic methods - if exposed to a risk factor of interest (ever, never).

3. Qualitatively compare calculated odds ratios generated by a single reviewer using H&E only, H&E+p16, or H&E+p16 and Ki67.

“Overall” kappa estimates – 0.46 (ALTS 2001) 0.49 (Horn 2008) 0.61 (Ceballos 2008)

Kappa estimates by grade of dysplasia – 0.57 0.38 0.74(Cai 2007)

www.cancerquest.org

Sensitivity Specificity PPV NPV

Pap smear 29-56% (low) 97-100% (high) 25-80% (low) 82%*

Colposcopy 95% (high) 64% (low) 40% 98%*

Colposcopic biopsy (H&E)

Moderate Moderate Problematic High?*

p16 IHC Moderate High Problematic High*

p16

Use of concurrent p16 in diagnosis of CIN has been demonstrated to improvekappa from 0.49 to 0.64. (Horn 2008)

Estimates of reproducibility in p16 assessment range between 0.74 and 0.91.(Klaes 2002, Agoff 2003)

Feng, W. et al. Modern Pathology. (2007) 20, 961-966.

Immunohistochemical Markers

Ki-67 is a non-specific marker of cellular proliferation.

Assessment of Ki-67 staining is moderately reproducible (kappa =0.70). (Agoff 2003)

Methods

Retrospective cohort of 440 women with any grade of CIN on colposcopic biopsy between Jan 1997 and Dec 2002.

Clinical parameters extracted from EMR and pharmacy records included:Age, parity, family income, education, h/o quadrivalent HPV vaccine, medical comorbidities, medication history

Random sample n=252 (84 each grade CIN)

Powered only for reproducibility aim.

Diagnose H&E slide

Non-CIN CIN

p16

CIN 1 CIN 2 or 3

Ki67

CIN 2CIN 3

For each caseBy each reviewer (TM and RK)Double blinded

Adolescents

Results

p16 improves precision for CIN 2p16 & Ki67 improve precision for CIN 1 – is the improvement worth the extra $$Role for IHC alone? Not current standard of care.

CIN 1 CIN 2 CIN 3

H&E only H&E only H&E only

TM vs. RK 0.6406 (0.0617) 0.4041 (0.0629) 0.6756 (0.0622)

Kaiser vs. TM 0.5254 (0.0623) 0.2162 (0.0630) 0.4736 (0.0620)

Kaiser vs. RK 0.3086 (0.0666) 0.1882 (0.0630) 0.4852 (0.0630)

H&E plus p16 H&E plus p16 H&E plus p16

TM vs. RK 0.6912 (0.0603) 0.4783 (0.0626)* 0.6678 (0.0621)

H&E plus p16 and Ki67 H&E plus p16 and Ki67 H&E plus p16 and Ki67

TM vs. RK 0.7453 (0.0615)* 0.5204 (0.0629)* 0.5997 (0.0605)

ResultsKaiser TM H&E TM p16 TM

H&E+p16TM Ki67 TM

H&E+bothRK H&E RK p16 RK

H&E+p16RK Ki67 RK

H&E+bothSensitivity 87.79% 80.81% 93.02%* 94.19%* 31.98% 97.09%* 88.37% 95.93%* 97.67%* 43.02% 97.67%*

Specificity 78.75% 86.25% 100%* 100%* 98.75%* 95.00%* 72.50% 71.25% 68.75% 96.25%* 68.75%

PPV 89.88% 92.67% 100%* 100%* 98.21%* 97.66%* 87.36% 87.77% 87.05% 96.10%* 87.05%

NPV 75.00% 67.65% 86.96%* 88.89%* 40.31% 93.83%* 74.36% 89.06%* 93.22%* 44.00% 93.22%*

CIN 1 CIN 2 CIN 3

H&E only H&E+p16 H&E only H&E+p16 H&E only H&E+p16

Sensitivity 52.38% 52.38% 76.06% 100%* 92.50% 100%*

Specificity 87.30% 100%* 84.62% 100%* 75.00% 100%*

PPV 57.89% 100%* 96.43% 100% 98.67% 100%

NPV 84.62% 86.30% 39.39% 100%* 33.33% 100%*

10.1 0.5 5 10Odds Ratio

Risk Factors for CIN2+Based on old H&E “gold standard diagnosis”

v Smoking (1.2-1.7) (McIntyre, 2005)

Low Education (1.2-2.4) (Khan, 2005)Black Race (0.4-0.9) (Khan, 2005)

Parity (0.8-1.07) (Belinson, 2008)

Contraceptives (0.9-1.1) (Castle, 2005)

Age (0.8-2.2) (Belinson, 2008)

HPV titre (2.5-65.0) (Song, 2006)

10.1 0.5 5 10Odds Ratio

Risk Factors for CIN2+Based on old H&E “gold standard diagnosis”

v Smoking (1.2-1.7) (McIntyre, 2005)

Low Education (1.2-2.4) (Khan, 2005)Black Race (0.4-0.9) (Khan, 2005)

Parity (0.8-1.07) (Belinson, 2008)

Contraceptives (0.9-1.1) (Castle, 2005)

Age (0.8-2.2) (Belinson, 2008)

HPV titre (2.5-65.0) (Song, 2006)

Biostatistics

Variable Kaiser TM H&E only

TM H&E+p16

TM H&E+p16/Ki67

RK H&E only

RK H&E+p16

RK H&E+p16/Ki67

Highest Dx

Age (<30, >30) 0.909 0.892 0.84 0.748 0.784 0.758 0.741 0.738Family income (<$45K/yr, >$45K/yr)

0.539* 0.892 0.585* 0.589* 0.61* 0.607* 0.618* 0.569*

Education level(<HS grad, >HS grad)

0.477* 0.650 0.630 0.70 0.571 0.557* 0.600 0.583

Gravidity (0, >1)

0.787 0.993 0.928 0.82 0.947 0.888 0.874 0.987

H/o STI? (Never, ever)

1.28 1.12 1.25 1.19 1.05 1.14 1.13 1.53

Vaginal hormone use?(Never, ever)

0.371* 0.191* 0.157* 0.136* 0.176* 0.159* 0.157* 0.165*

Results: Odds of CIN2+ by dx method

OCP use? (Never, ever)

0.769 1.17 1.02 1.13 1.11 1.18 1.20 1.15

Race (White, all other)

0.949 0.94 0.811 0.877 0.804 0.842 0.828 0.703

Parity (0, >1) 0.84 0.842 0.924 0.784 0.824 0.980 0.966 1.05

H/o smoking? (Never, ever/unknown)

0.82 0.934 0.866 0.887 1.08 0.798 0.811 0.991

Conclusions & Future Directions

1. Use of p16 and Ki-67 improves precision and accuracy in diagnosis of CIN.

2. Use of Ki67 may not provide sufficient improvements to justify additional cost.

3. Improved outcome classification due to use of p16 and Ki-67 hassignificant effect on the interpretation of observed effect measures.

4. Immunohistochemistry may have utility for improving precision and accuracyin diagnosis in other organ systems. Evaluation of this potential may contributeto improvements in classification of other cancer outcomes.

Acknowledgments

MPH Thesis Committee: OCTRI Assistance:Michelle Berlin, MD, MPH Cindy Morris, PhD, MPHTerry Morgan, MD, PhD Mary Samuels, MDTomi Mori, PhD Ethan Siefert

Karen McCrackenAnnette Vu

Laboratory Assistance: Kaiser Collaborators:Carolyn Gendron & Corless Lab Rob Krum, MDMorgan Lab Dan Sapp

Denise SchwarzkopfParker Pettus

“one certainly cannot predict future events exactly if one cannot even measure the present state of the universe precisely!”

Stephen Hawking, A Brief History of Time