CCEB

The Analysis and Interpretation of

Pain Clinical Trial Outcomes:Enhancing Understanding

John T. Farrar, MD, PhDUniversity of Pennsylvania

CCEB

Surrogate Outcomes

Only three “real” outcomes

• Birth• Death• Quality of life

CCEB

Changing the State of the Brain

What do you see?

CCEB

Why Do We Care

• RCTs - important for most medical therapy• Did not need an RCT for introduction of

penicillin–Pneumococcal pneumonia–No penicillin - Last week 9/10 people died–With penicillin – This week 1/10 people died

• Corollary – if you identify the right group, measurement and design issues statistics will be less controvertial

CCEB

Outline of the Presentation

• Measurement must be appropriate

• Handling of missing data is important

• Part 1: How do patients report pain

• Part 2: Analysis

• Part 3: Interpretation

CCEB

Pain is a Subjective Experience

• No “objective” direct measure• Not easy to relate to an underlying

neurologic process in an individual• Depend on subjects to accurately

report their experience

• Creates inter-person variation in the reporting of pain that is unavoidable

• Creates observer discomfort about the validity of the measure

CCEB

Pain Measures - Intensity Scales

0__1__2__3__4__5__6__7__8__9__10

|____________________________________________|

| |Least Worst

None Mild Moderate Severe Excruciating

Intra-person reliability – Good

Inter-person reliability – Poor

CCEB

How do Patients DecideIf a Treatment is Useful

• Does the treatment make my symptoms better now?

• Are there any side-effects?• Is the pain relief “good enough”?

>>>> Am I better overall?

>>>> Should I take something else?

CCEB

Global Rating of Quality of Life

Overall how would you rate your quality of life: over the last ______:

0 1 2 3 4 5 6 7 8 9 10Worst BestIt can be it can

be

CCEB

Global Change in Quality of Life

How has your quality of life changed over the last ______: (or - since the last _____:)

Very Much Worse

Much Worse

A little worse

Very Much better

MuchBetter

A little better

No change

Another View on Scales

CCEB

How Do Patients Use a Numeric Scale(Acute Pain)

• Study data: Randomized clinical trial of oral trans-mucosal fentanyl versus placebo

• Method: Re-analysis of data set stratified on baseline pain intensity score

• Population: 89 cancer pain patients with acute breakthrough pain

• Results =>

CCEB

Data Collection Instrument

• Baseline – Pain Intensity 1_2_3_4_5_6_7_8_9_10

• At 15, 30, 45 and 60 minutes– Pain Intensity 1_2_3_4_5_6_7_8_9_10– Pain Relief 0 (none) 1(slight) 2(mod.) 3(lots) 4(comp.)

• Second rescue medication - Time________– Overall Performance

» 0 (none) 1(slight) 2(moderate) 3(lots) 4(complete)

CCEB

-10

-8

-6

-4

-2

0

Poor Fair Good Very Good Excellent

Ra

w P

ID

4

5

6

7

8

9

10

Raw Change in Pain Intensity Compared to Global Performance Scale

Global Performance Scale

CCEB

-10

-8

-6

-4

-2

0

Poor Fair Good Very Good Excellent

Ra

w P

ID

4

5

6

7

8

9

10-100%

-80%

-60%

-40%

-20%

0%

Poor Fair Good Very Good Excellent

% P

ID

Percent Change in Pain Intensity Compared to Global Performance Scale

Global Performance Scale

CCEB

How Do Patients Use a Numeric Scale(Chronic Pain)

• Study data - RCTs of pregabalin in multiple diseases

• Method – Compared measured pain intensity (0-10 NRS) and patients global impression of change (PGIC)

• Population - Data on 2,724 subjects from 10 clinical trials of diabetic neuropathy (3), postherpetic neuralgia (3), chronic low back pain (2), fibromyalgia (1) and osteoarthritis (2).

Reduction of Pain Diary Scores from Baseline to Endpoint

-8

-7

-6

-5

-4

-3

-2

-1

0

1

2

Very Much Worse /Much Worse

Minimally Worse No Change Minimally Improved Much Improved Very MuchImproved

PGIC Category

Raw

Ch

ang

e S

core BP = 4

BP = 5

BP = 6

BP = 7

BP = 8

BP = 9

Percent Reduction of Pain Diary Scores from Baseline to Endpoint

-80

-70

-60

-50

-40

-30

-20

-10

0

10

20

30

Very Much Worse /Much Worse

Minimally Worse No Change Minimally Improved Much Improved Very MuchImproved

PGIC Category

Per

cen

t C

han

ge

Sco

re

BP = 4

BP = 5

BP = 6

BP = 7

BP = 8

BP = 9

CCEB

Clinically Important Differencesfor the 0-10 NRS

• Used the global response levels as the metric of a clinical importance response

• Compared change in 0-10 NRS measure over time to this standard

• Determined the clinically important change cut-off by calculating:

–Sensitivity, specificity and accuracy

–Receiver Operator Characteristic (ROC) analysis

CCEB

Studies of Duloxetine

• Secondary analysis of 5 studies–Diabetic neuropathy – 3–Fibromyalgia – 2

• Total number of patients – 1600• Study period – 12 weeks• Pain measures – 0-10 NRS

–Worst, least, average• Patient global impression of change

Imp

rove

men

t

BP

I Ave

rage

Pai

n P

erce

ntag

e C

hang

e S

core

-80-70-60-50-40-30-20-10

01020304050607080

Study 1: DPNPStudy 2: DPNPStudy 3: DPNPStudy 4: FMStudy 5: FM

-80-70-60-50-40-30-20-10

01020304050607080

Duloxetine (N=1077)

Placebo (N=533)

-80-70-60-50-40-30-20-10

01020304050607080

Male (N=636)Female (N=974)

-80-70-60-50-40-30-20-10

01020304050607080

18-49 (N=434)50-59 (N=547)60-69 (N=431)70+ (N=198)

A B

C D

Very much worse/Much worse

A little worse

No change

A little better

Much better

Very much better

Very much worse/Much worse

A little worse

No change

A little better

Much better

Very much better

Very much worse/Much worse

A little worse

No change

A little better

Much better

Very much better

Very much worse/Much worse

A little worse

No change

A little better

Much better

Very much better

Very much worse/Much worse

A little worse

No change

A little better

Much better

Very much better

Very much worse/Much worse

A little worse

No change

A little better

Much better

Very much better

Very much worse/Much worse

A little worse

No change

A little better

Much better

Very much better

Very much worse/Much worse

A little worse

No change

A little better

Much better

Very much better

PGI CategoryPGI Category PGI CategoryPGI Category

PGI CategoryPGI Category PGI CategoryPGI Category

CCEB

Receiver Operator Response Curve Percentage Pain Intensity Difference

Clinically Important Values

BPI average pain

(N=1610) Model

BPI average pain

(N=1610)

BPI worst pain

(N=1612)

Raw change Very much better -3.5 -4.0

Raw change Much or very much better

-2.5 -3.0

Raw change A little, much, or very much better

-2.0 -2.0

Percentage change

Very much better -51% -51%

Percentage change

Much or very much better

-34% -34%

Percentage change

A little, much, or very much better

-23% -21%

CCEB

Part 1: Conclusion

• Patients use the 0-10 NRS scale primarily as a percent scale and is best analyzed as a percent change from baseline pain

• A 30-35% improvement on the 0-10 NRS pain intensity scale is a reasonable cut-off point for a clinically important change

CCEB

Two Groups Randomized:Both centered at 20% change at end of the study

0

2

4

6

8

10

12

14

16

-50%

-43%

-35%

-28%

-20%

-13% -5% 3% 10%

18%

25%

33%

40%

48%

55%

63%

70%

78%

85%

93%

100%

108%

115%

123%

130%

138%

145%

Control

Combined Treatment

Percent change from baseline

CCEB

Actually Bimodal Distribution

0

2

4

6

8

10

12

14

16

-50%

-43%

-35%

-28%

-20%

-13% -5% 3% 10%

18%

25%

33%

40%

48%

55%

63%

70%

78%

85%

93%

100%

108%

115%

123%

130%

138%

145%

Control

Non-Responders

Combined Treatment

Percent change from baseline

CCEB

Actually Bimodal Distribution

0

2

4

6

8

10

12

14

16

-50%

-43%

-35%

-28%

-20%

-13% -5% 3% 10%

18%

25%

33%

40%

48%

55%

63%

70%

78%

85%

93%

100%

108%

115%

123%

130%

138%

145%

Control

Responders

Non-Responders

Combined Treatment

Percent change from baseline

CCEB

0

2

4

6

8

10

12

14

16

-0.5

-0.4

-0.4

-0.3

-0.2

-0.1 -0

0.03 0.1

0.18

0.25

0.33 0.4

0.48

0.55

0.63 0.7

0.78

0.85

0.93 1

1.08

1.15

1.23 1.3

1.38

1.45

Control

Responders

Non-Responders

Combined Treatment

CCEB

Study Efficiency Mean vs Dichotomous Analysis

0

2

4

6

8

10

12

14

16

-50

%

-43

%

-35

%

-28

%

-20

%

-13

%

-5%

3%

10

%

18

%

25

%

33

%

40

%

48

%

55

%

63

%

70

%

78

%

85

%

93

%

10

0%

10

8%

11

5%

12

3%

13

0%

13

8%

14

5%

Control

Responders

Non-Responders

Combined Treatment

Efficiency = 1.145 (T-test N=30, Chi-sq N=26)

Mean of Control Group and Low Probability Responders = 15% Mean of the Treated Group = 34%; Cut-off = 33%

CCEB

Group Mean Results – PIDOral Transmucosal Fentanyl Citrate (OTFC)

Pain Intensity Differences (with imputed values)

0

1

2

3

4

0 15 30 45 60

Minutes

Better^|||||||^

OTFC

Placebo

p<.001 at all time points

Farrar JT, et al Oral transmucosal fentanyl citrate: randomized, double-blinded, placebo-controlled trial for treatment of breakthrough pain in cancer patients. Journal of the National Cancer Institute 1998; 90(8): 611-6

CCEB

OTFC Study Outcomes: Relative Risk Comparison

At 60 minutes

PID >33% %Max TotPAR Relative Risk 1.89 1.56 Conf. Interval 1.59 – 2.17 1.30 – 1.93

p-value <0.0001 <0.0001

PR >1 Perf >1

Relative Risk 1.73 1.56 Conf. Interval 1.42 – 1.95 1.34 – 1.95

p-value <0.0001 <0.0001

OTFC Looked at Density Plots

Density Function

CumulativeDistributionFunction

CCEB

OTFC Placebo Proportion of Responders (at different cut off points - for 30 minutes)

Pro

port

ion

of R

espo

nder

s

Percentage Pain Intensity Difference

Cumulative Distribution of Responders Graph

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

80.0%

90.0%

100.0%

0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0% 100.0%Percent of Pain Intensity Difference

OTFC #1

Placebo

CCEB

Mean Value Does Not Provide a Unique Answer to the Clinical Question

• Mean value for the change in pain intensity over time is 10%. This would be observed if:

• 1) every patient in the treatment group improved by 10%, or

• 2) if 50% of the treatment group got better by 20% and 50% had no improvement, or

• 3) if 50% of the treatment group got better by 40% and 50% got worse by

20%.

CCEB

FDA Primary Data Analysis

• Data source–Neuropathic pain RCTs (n=15)– Indications

» Post-herpetic neuralgia (n=7)» Diabetic peripheral neuropathy (n=8)

–Pharmaceuticals» Pregabalin (n=11)» Gabapentin (n=2)» Duloxetine (n=2)

–Primary outcome measure» Change in 0-10 NRS pain score

Representative Data

CCEB

Method

• Each RCT was analyzed using absolute and percent change of the mean pain score.

• List of the analytic methods compared for each trial for between group analyses (active treatment vs. placebo)– T-test– Wilcoxon rank sum test– Kolmogorov-Smirnov test – ROC based - AUC comparison– Ordinal logistic regression– Log rank

T = T-test with equal varianceRS = Wilcoxon rank sum testROC = AUC comparison

KS = Kolmogorov-Smirnov OL = Ordinal logistic regressionLR = Log rank

T = RS ROC KS OL LR

pbo - - - - - -p150 0.2445 0.1865 0.1847 0.5362 0.2230 0.2664

rank 4 2 1 6 3 5p600 0.0003 0.0006 0.0003 0.0003 0.0011 0.0001

rank 2 5 4 3 6 16 7 5 9 9 6

pbo - - - - -p150 0.1542 0.1101 0.1071 0.2919 0.1055 0.0795

rank 5 4 3 6 2 1p600 0.0002 0.0003 0.0002 0.0005 0.0004 0.0000

rank 3 4 2 6 5 18 8 5 12 7 2

dif

total rank

pctchg

Test

total rank

CCEB

Rank Totals

T = T-test with equal varianceRS = Wilcoxon rank sum testROC = AUC comparisonKS = Kolmogorov-Smirnov OL = Ordinal logistic regressionLR = Log rank

T = RS ROC KS OL LR

rank totals 117 109 64 165 108 130

rank totals 105 127 79 156 99 125

Test

dif

pctchg

CCEB

Conclusions

• Tools for measuring pain have high inter-person variability and lower intra-person variability

• Mean values do not provide a unique answer to the clinical question of how many people get better

• Responder analysis accurately reflect the number of people in each treatment group that reach a level of change in that study

CCEB

Conclusions (cont)

• To the degree that the test group is an accurate representation of the general population the response rates in the treated group will reflect what the clinician is likely to see, regardless of the reason for the response.

• Both mean value and the responder analysis provide useful information and should be presented.

CCEB

THANK YOU

Research Group

• Chris Rowan• Kevin Haynes• Andrea Troxel• Brian Strom• Rosemary Polomano

Additional Collaborators

• Robert Dworkin• Dennis Turk• Nathaniel Katz• Michael Rowbotham• Russel Portenoy• John Messina• Michael Poole• Mitchell Max• Jesse Berlin