v.: 9/7/2007 AC Submit 1

Statistical Review of the Observational Studies of Aprotinin Safety

Part I:Methods, Mangano and Karkouti Studies

CRDAC and DSaRM MeetingSeptember 12, 2007

Mark Levenson, Ph.D.Quantitative Safety & Pharmacoepidemiology Group

Office of Biostatistics

2

Acknowledgements to Dr. Mangano and Dr. Karkouti

3

Outline

1. Review Objectives

2. Statistical Methods

3. Mangano Review

4. Karkouti Review

5. Summary

4

1. Review Objectives

1. To confirm the reported findings based on investigators’ methods

2. To evaluate the statistical robustness of the findings

– FDA analyzed the 3 studies• Same methods applied to all three studies• Robust • Diagnostics

5

Outline

1. Review Objectives

2. Statistical Methods

3. Mangano Review

4. Karkouti Review

5. Summary

6

Propensity Scores (PS)• Adjust for differences in baseline risk factors between

two treatment groups (Treatment A and Treatment B)

• Definition: Probability of assignment for a patient to Treatment A versus Treatment B based on measured risk factors

• Intuition: – Suppose Treatment A patient and Treatment B

patient have the same PS.– Comparisons between these two patients are fair

7

Propensity Scores Practice

• Balance: similarity of distributions of a risk factor between treatment groups

• PS methods cannot account for unmeasured confounders

• The PS are estimated based on statistical modeling– Diagnostics important

8

Propensity Scores Practice (Cont.)

• Estimating treatment effects using PS– Matching (Karkouti)– Stratification (FDA)– Multivariate regression (Mangano, i3)

9

FDA Analysis Methods

• Pre-specified

• Propensity scores with stratification was used to adjust for baseline risk factors

• Medical, epidemiological, and statistical expertise was used to choose risk factors

• Diagnostics (analytical and graphical) were used to evaluate balance and explore findings

10

Outline

1. Review Objectives

2. Statistical Methods

3. Mangano Review

4. Karkouti Review

5. Summary

11

Mangano Study: Key Points

• Prospectively specified – Inclusion criteria– Outcome definitions– Subgroups– Analysis methods

• In-hospital outcomes (NEJM)• Long-term mortality follow-up outcomes (JAMA)• 7 of 69 centers did not participate in the long-

term follow-up

12

Mangano Study: Key Points Analysis Methods

• Multivariate regression with and without propensity score as a covariate– Logistic for in-hospital outcomes– Cox PH for long-term mortality

• Propensity score for any active agent versus no agent for full analysis group

• No adjustment for geographical differences

13

Mangano Study: Patients by Geographical Region and Treatment Group

Region

No Agent

N=1374

%

Aprotinin

N=1295

%

Amino-caproic

N=883

%

Tran-examic

N=822

%

Europe 57 69 0 49

North America

24 29 96 29

Other 19 1 4 22

14

Mangano Study: Long-Term Follow-up

No AgentN=1374

%

AprotininN=1295

%

Completed 5-year follow-up or died

73 83

No post-hospital follow-up

10 1

Lost to follow-up in the post-hospital period

17 16

15

Mangano Study: Demographic Factors

Characteristic No AgentN=1374

AprotininN=1295 P-Value

Age (mean ± sd) 63 ± 10 65 ± 9 <.001

Male (%) 81 79 0.135

African American or Hispanic (%) 4 4 0.542

16

Mangano Study: Selected Baseline Risk Factors

Characteristic

No AgentN=1374

(%)

AprotininN=1295

(%) P-Value

Surgical: CABG + Other 11 19 <.001

Surgical: Non-Elective 21 15 <.001

History of liver disease 8 12 <.001

History of renal disease 13 19 <.001

Previous sternotomy 3 13 <.001

Preop: Creatinine >1.3 mg/dL 14 15 0.338

Preop: Ejection fraction ≤ 44% 18 15 0.071

Preop: MI 15 16 0.863

17

Mangano Study:Study Reported Findings and Methods

• Primary findings of NEJM and JAMA based on investigators’ methods reproduced

• Imbalances in baseline risk factors and geographical regions between aprotinin and no-agent groups after PS adjustment

• Lack of overlap in propensity score distributions between aprotinin and no-agent groups

18

FDA Analysis Results

19

Mangano Study: FDA AnalysisBaseline Risk Factors

Before and After PS Adjustment

Before PS Adjustment After PS Adjustment

NoAgent

%Aprotinin

% P-Value

NoAgent

%Aprotinin

% P-Value

Surgical: CABG + Other 11 19 <.001 15 16 0.543

Surgical: Non-Elective 21 15 <.001 18 19 0.651

History of liver disease 8 12 <.001 10 10 0.913

History of renal disease 13 19 <.001 16 16 0.849

Previous sternotomy 3 13 <.001 6 8 0.029

Preop: Elev. Creatinine 14 15 0.338 14 14 0.943

Preop: Eject. Fr. ≤ 44% 18 15 0.071 16 16 0.878

Preop: MI 15 16 0.863 15 15 0.745

20

Mangano Study: FDA AnalysisIn-Hospital Outcome Adjusted Estimates

Aprotinin vs. No Agent*

OutcomeNo Agent

(%)Aprotinin

(%)

Risk RatioAprotinin/No Agent

(95% CI)

Renal Composite 4.8 7.8 1.63 (1.03, 2.60)

Renal Failure 2.5 5.1 2.05 (1.05, 3.99)

Renal Dysfunction 4.3 5.4 1.26 (0.76, 2.11)

Cardiovascular Composite 19.5 22.2 1.14 (0.94, 1.38)

Myocardial Infarction 14.8 16.4 1.10 (0.88, 1.39)

Congestive Heart Failure 8.4 8.8 1.05 (0.75, 1.47)

Stroke 2.0 2.8 1.36 (0.70, 2.64)

Death (in-hospital) 4.6 4.2 0.91 (0.54, 1.53)

*Analysis based on 1307 no agent patients and 1222 aprotinin patients

21

Mangano Study: FDA AnalysisRenal Composite

0.0

00

.05

0.1

00

.15

0.2

00

.25

Propensity Score Strata

Re

na

l Co

mp

osi

te

1 2 3 4 5 6 7 8 9 10

No AgentAprotinin

205 187 164 160 134 130 105 104 72 4647 66 89 93 119 123 148 149 181 207

n=

22

Mangano Study: FDA AnalysisLong-Term Mortality Adjusted Estimates

Aprotinin vs. No Agent*

OutcomeNo Agent

(%)Aprotinin

(%)

Risk RatioAprotinin/No Agent

(95% CI)

6 Weeks 5.2 4.8 0.93 (0.57, 1.51)

6 Months 6.4 7.0 1.10 (0.73, 1.68)

1 Year 7.1 8.2 1.16 (0.79, 1.71)

2 Years 8.4 10.7 1.27 (0.90, 1.79)

3 Years 9.7 13.0 1.34 (0.98, 1.83)

4 Years 11.4 15.9 1.39 (1.05, 1.84)

5 Years 14.3 18.1 1.26 (0.98, 1.62)

*Analysis based on 1307 no agent patients and 1222 aprotinin patients

23

Mangano Study: FDA AnalysisLong-Term Mortality Adjusted Estimates

0.0

00

.05

0.1

00

.15

0.2

00

.25

Years

Mo

rta

lity

0 1 2 3 4 5

No AgentAprotinin

1164 1134 1096 1050 1007 909 881

1189 1144 1085 1015 954 888 849N Risk

24

North American Subgroup

25

Mangano Study: FDA Analysis by Region and Treatment Group

0.0

0.2

0.4

0.6

0.8

1.0

Pro

pe

nsi

ty S

core

No AgentAprotinin

Europe North America

26

Mangano Study: FDA AnalysisNorth America

In-Hospital Outcome Adjusted EstimatesAprotinin vs. Aminocaproic*

OutcomeAmino.

(%)Aprotinin

(%)

Risk RatioAprotinin/Amino.

(95% CI)

Renal Composite 2.4 9.4 3.90 (1.98, 7.70)

Renal Failure 0.6 5.3 9.17 (3.10, 27.15)

Renal Dysfunction 2.0 6.7 3.39 (1.54, 7.44)

Cardiovascular Composite 17.7 22.5 1.28 (0.94, 1.74)

Myocardial Infarction 12.1 14.9 1.23 (0.83, 1.81)

Congestive Heart Failure 7.5 10.5 1.40 (0.84, 2.35)

Stroke 1.5 3.2 2.10 (0.76, 5.81)

Death (in-hospital) 2.1 4.7 2.21 (0.88, 5.52)

*Analysis based on 789 aminocaproic patients and 342 aprotinin patients

27

Mangano Study: FDA AnalysisNorth America:

Long-Term Mortality Adjusted Estimates Aprotinin vs. Aminocaproic*

OutcomeAmino.

(%)Aprotinin

(%)

Risk RatioAprotinin/Amino.

(95% CI)

6 Weeks 2.4 4.7 1.91 (0.82, 4.47)

6 Months 3.5 7.3 2.11 (1.04, 4.28)

1 Year 5.1 9.9 1.96 (1.10, 3.47)

2 Years 7.3 13.7 1.89 (1.19, 3.00)

3 Years 10.0 18.4 1.84 (1.23, 2.73)

4 Years 15.1 21.9 1.45 (1.03, 2.04)

5 Years 19.8 24.6 1.24 (0.92, 1.67)

*Analysis based on 789 aminocaproic patients and 342 aprotinin patients

28

Mangano Study: Review Summary

• Renal outcomes (particularly renal failure) effect in a range of patients and in North American region subgroup

• Effects for cardiovascular, cerebrovascular, and in-hospital death outcomes not statistically demonstrated

• Long-term mortality effects in a range of patients and in North American region subgroup

29

Outline

1. Statistical Review Objectives

2. Statistical Methods

3. Mangano Review

4. Karkouti Review

5. Summary

30

Karkouti Study: Key Points

• Retrospective study of 5 years of patient data from a single center

• Patient population: Cardiac surgery (CABG and non-CABG) with cardio-pulmonary bypass

• Aprotinin used for high risk patients, tranexamic acid used for other patients

• Used propensity scores with 1-1 matching– 449 of 586 aprotinin patients matched

31

Karkouti Study: Demographic Factors

Characteristic

TranexamicAcid

N=10251AprotininN=586 P-Value

Age (mean ± sd) 63 ± 12 55 ± 17 <.001

Male (%) 75 65 <.001

32

Karkouti Study: Selected Baseline Risk Factors

Characteristic

TranexamicAcid

N=10251(%)

AprotininN=586

(%) P-Value

Surgical: Not CABG only 33 89 <.001

Surgical: Non-Elective 8 19 <.001

Previous sternotomy 5 61 <.001

Preop: Creatinine abnormal 19 26 <.001

Preop: Ejection fraction <40% 20 23 0.148

Preop: MI 17 7 <.001

33

Karkouti Study: Study Reported Findings and Methods

• Primary findings using investigators’ methods reproduced

• Observed risk factors balanced with propensity score matching approach

34

Karkouti Study: FDA Analysis

• A subgroup of patients with overlap in propensity scores was defined to enable treatment comparison

• Subgroup contained – 553/586 (94%) of the aprotinin patients– 3759/10251 (37%) of tranexamic patients

• Baseline risk factors more similar between treatment groups in subgroup than full group

35

Karkouti: FDA AnalysisAnalysis Subgroup, Baseline Risk Factors

Before and After PS Adjustment

Before PS Adjustment After PS Adjustment

Tran.%

Aprotinin% P-Value

Tran.%

Aprotinin% P-Value

Surgical: Not CABG only 83 92 <.001 83 83 0.873

Surgical: Non-Elective 16 18 0.137 16 19 0.366

Previous sternotomy 14 64 <.001 19 22 0.126

Preop: Elev. Creatinine 3 5 0.010 3 3 0.950

Preop: Eject. Fr. <40% 21 22 0.426 21 19 0.693

Preop: MI 11 7 0.003 10 16 0.036

36

Karkouti Study: FDA AnalysisTreatment Effect Estimates

Aprotinin vs. Tranexamic Acid*

Outcome

FDA AnalysisRisk Ratio

Aprotinin/Tran.

Matched-PairOdds Ratio

Aprotinin/Tran.

Renal dysfunction 1.53 (1.11, 2.12)

Renal failure 1.38 (0.86, 2.23) 1.85 (0.94, 3.63)

Myocardial Infarction 1.42 (0.71, 2.83) 1.22 (0.51, 2.95)

Stroke 1.72 (0.93, 3.19) 1.15 (0.55, 2.43)

Death (in-hospital) 1.18 (0.79, 1.76) 0.90 (0.54, 1.51)

*Analysis based on 3759 tran. patients and 553 aprotinin patients

37

Karkouti Study: FDA AnalysisRenal Failure

0.0

00

.05

0.1

00

.15

0.2

00

.25

Propensity Score Strata

Po

st-O

pe

rativ

e R

en

al F

ailu

re

1 2 3 4 5

Tran. AcidAprotinin

852 838 821 768 48010 25 41 95 382

n=

38

Karkouti Study: Review Summary

• Renal dysfunction effect statistically significant

• Some evidence for renal failure effect

• Effects for myocardial infarction, stroke, and in-hospital death outcomes not statistically demonstrated

39

Outline

1. Review Objectives

2. Statistical Methods

3. Mangano Review

4. Karkouti Review

5. Summary

40

Summary

• Evidence for renal effect, including renal failure consistent

• Effects for cardiovascular, cerebrovascular, and in-hospital death outcomes not statistically demonstrated

• Evidence for long-term mortality effect • Potential for unadjusted confounders

between the treatment groups which may bias the treatment effect estimates