. .i ::....,

THE COX REGRESSION MODEL. RANDOM CENSORINGAND LOCALLY OPTIMAL RANK TESTS

by

Pranab Kumar Sen

Department of BiostatisticsUniversity of North Carolina at Chapel Hill

Institute of Statistics Mimeo Series No. 1376

February 1982

THE COX REGRESSION MODEL, RANDOM CENSORINGAND LOCALLY OPTIMAL RANK TESTS

by

Pranab Kumar Sen

University of North Carolina, Chapel Hill

Key Words and Phrases: Asymptotic efficiency; covariates; hazard function;

LMPR test; local optimality; log-rank test; survival function;

withdrawal.

ABSTRACT

Conditions on the hazard functions under which the usual log-rank

test remains locally optimal for the Cox regression model under random

censoring (withdrawal) are examined. In the light of these, the asymptotic

efficiency results pertaining to the Cox partial likelihood statistic

and the log-rank statistic are studied.

1. INTRODUCTION

In the Cox (1972) regression model for survival data, it is assumed

that the ith subject (having survival time Yi and a set of covariates

Zi ... (Zil'···'Zi )', for some p > 1) has the hazal'd l'ate (given Z... zi)- p - -~ ....

(1.1)

where hO(t), the hazrad rate for ~i .. 0, is an unknown, arbitrary,00

nonnegative function (for which f hO(t)dt- +00) and a - (B1, ••• ,a )'o - p

parameterizes the regression of survival times on the covariates. If

the conditional distl'ibution fUnction (d.f.) and, its complement, the

sUl'Vival function (s.f.) of Yi , given ~1 = ~i' are denoted by Fi(yl~i)'

-2-

and Fi(yl~i)' respectively. then by (1.1). we have

_ _ exp(~'~i)

Fi(yl~i) • [FO(Y)] • i=l •••••n. (1.2)

where

(1.3)

In the particular case of p = 1 and binary zi (assuming the values °or 1).

(1.2) reduces to the Lehmann (1953) model. so that the Cox model includes

the Lehmann model as a special case. Also. for scalar zi and B. it

follows from a general theorem in Hajek and Sidak (1967. pp. 70-72)

that for testing HO: B= °against non-null Bclose to 0, a locally most

powerful rank (LMPR) test is based on the statistic

(1.4)

where

scores

Rni is the rank of Yi among Y1, •.• ,Yn , for i=l •••• ,n. and the

° °a (l), ••. ,a (n) are defined byn n

aO(k) = -1 - E{log(l-U k)}n n,

k -1(= -1 + Lj _1(n- j +1) ), for k=l, ••• ,n, (1.5)

where U 1 <••• < U are the random variables of a sample of size nn, n.n

from the uniform (0,1) distribution. The scores in (1.5) are known as the

olog-rank scores and T as the log-rank statistic or the (generalized)n

Savage statistic. When p ~ 1. we may define

and

o n - 0 - -1 nT = Li l(Zi-Z )a (R i)' Z = n Li 1Z'~n = _ -n n n ~n = ~~

(1.6)

(1. 7)

(1.8)

-3-

oThen Ln provides a loaaZly maximin rank test for HO: e = 0 against non-null

e close to O.

In survival analysis. random censoring due to withdrawals is not

uncommon. Here. we conceive of a set W1•••••Wn of independent random

variables (censoring times) and assume that the Wi and Yi are independent.

The ovservable r.v.'s are then

i > 1 • (1.9)

If G and G be respectively the d.f. and s.f. for the Wi (assumed to be

identically distributed) and if Si(xl~i) and Si(xl~i) be respectively

the d.£. and s.L of Xi' given ~i .. ~i (ignoring 0i)' then by (1.2) and

(1.9),

Si(xl~i) .. G(x)Fi(xl~i)

_ _ exp(S'zi)= G(x) [FO(x)] - - ,i=1 •••••n. (1.10)

Note that under HO: ~ = 0, S1, ••• ,Sn are all the same (i.e., equal to

G FO)'so that if one ignores the information contained in o • (01, •••• 0 )',-n n

rank tests based on X1, ••• ,Xn are genuinely distribution-free (un?er HO).

o 0The test based on Ln or !n (replacing the Yi by Xi' 1 ~ i ~ n) is therefore

a valid test for HO: ~ • ~ under random censoring, though it may not be locally

optimal.

If g is the density function corresponding to the d.f. G. then the

hazard rate for the Wi is hG(t) = g(t)/G(t). so that the hazard rate for

the Xi (conditional on ~i = ~i) are

(1.11)

-4-

This, in general, vitiates the proportionality assumption in (1.1), and

hence, the log-rank test may not be locally optimal.

Note that the likelihood function (of the Xi and 0i' given ~i) is

given by

(1.12)

so that the joint distribution of the ranks and ° depends on the unknown-n

FO

and G, even under HO: ~ • O. To eliminate this problem, Cox (1972,

1975) considered a partial likelihood function (which takes into account

the indicator variables 0l""'On) and obtained some (non-rank) test

statistics which depend on the covariates ~i' i=l, ••• ,n in a more involved

way. Peto and Peto (1972) considered the two-sample problem [as a special

case of (1.10) with binary zi] and, under a somewhat different setup,

concluded that the log-rank statistic is LMPR even under random censoring.

As we shall see in Section 2 that under the model (1.10)-(1.11), the 10g-

rank statistic is not generally LMPR (or maximin), even for the special

case of the two-sample problem. For this reasom, we will investigate the

conditions (on the hazard rates hO(t) and hG(t) or equivalently on FO and

G) under which the log-rank statistic is locally optimal among the rank

based tests on X1""'Xn (gnoring ~n)' The question that naturally

arises about the gain in efficiency of the Cox procedure (due to incor-

poration of ~ ) over the log-rank procedure (ignoring °), and this will-n _n

be addressed here to 0 •

Section 2 is devoted to the study of locally optimal rank tests (for

testing HO: ~ = ~), under the model (1.10). Section 3 deals with the local

optimality of the log-rank test. Asymptotic efficiency results on the Cox

-5-

procedure are presented in Section 4. and the concluding section is devoted

to some general remarks.

2. LOCALLY OPTIMAL RANK PROCEDURES UNDER RANDOM CENSORING

Ignoring 0l' .••• On and based on the ranks of Xl •.•.• Xn • we like to

construct suitable tests for HO: ~ • 9. having some local optimality

properties.

Let si(xl~i) be the probability density function (p.d.f.) corresponding

to the d.f. Si(xl~i) in (1.10). for i·l •••••n. Then. by (1.10) and (1.11).

we have

log Si(xl~i) = log Si(xl~i) + log h~(x)

• log G(x) + log Fi(xl~i) + log h~(x)

= log G(x) + exp(~'~i)log FO(x) +

10g[hG(x) + eXP(~'~i)hO(x)]. 1 < i < n

(2.1)

which leads to the log-likelihood function (of the Xi given ~i = ~i'

ial ••.••n). ignoring 0, as-n

n - n -• ~i=llog G(Xi ) + ri.lexP(~'~i)log FO(Xi ) +

~~=llog[hG(Xi) + exp(~'~i)hO(Xi)] • (2.2)

Note that by (2.2)~

where

-6-

Further note that under HO: ~ m 2, sl(xl~l) • •.• = sn(xl~n) = So(x) ...

G(x)FO(x) and this does not depend on ~l' ... '~n. Thus, under HO'

X1

, .•. ,Xn

are independent and identically distributed random variables

(i.i.d.r.v.), so that R = (R 1, ••. ,R ), the vector of ranks of X1, ••• ,Xn~n n nn

among themselves, has the (discrete) uniform distribution over the set

*of n! possible permutations of (l, ••• ,n). Let then ~ = ~l + ~2' where

= log FO(x) 1_ ,0< u < 1, (2.5)SO(x)=l-u

(2.6)O<u<l,~2(u) ... 7T(X) 1_SO(x)=l-u

define the ordered r.v.'s U l' •.• 'U as in after (1.5), and letn, n,n

*Also, let a (k) = a l(k) + a 2(k), for ka 1, ••• ,n. Then, by (2.3),n n, n,

(2.4), (2.5), (2.6) and (2.7), we obtain that

* n * *EO{(ll/da)log L (a) I IR} = LZ.a (R i) ... T , say, (2.8)- n a=O n i=l-~ n n _n

where EO denotes the expectation under HO: a = O.

Note that by (2.4) and (2.6),

1~2 • ! ~2(u)du'" !7T(x)dSO(x)'" -J 7T(x)dSO(x)

o

= f FO(x)G(x)hO(x)dX

[by (1. 3) ]= -f FO(x)G(x)d log FO(x)

... flog FO(x)d.So(x) ... -J log FO(x)dSO(x)

1... -J ~ (u)du = - ~011

(2.9)

Further, by (2.7)

(2.10)

-7-

Hence, from (2.9) and (2.10), we have

*Thus, by (2.8) and (2.11), we may rewrite T as-n

(2.12)

At this stage, we may note that (i) hO(x)exp(~'~i)/{hO(x)exp(~'~i)+

hG(x)} is a bounded and continuous function of a, and (ii) for every x,

o ~ - log FO(x) = - log SO(x) + log G(x) ~ - log So (x) , where the right

hand side is square integrable (with respect to SO(x». Hence, for p - 1,

we may appeal directly to a theorem in Hajek and ~idtk (1967, p.71),

verify their basic conditions (1) and (2) (on page 70) and conclude that

*Tn is a LMPR test statistic for testing Ho: a • 0 (based on the Xi and

ignoring the 0i). For p ~ 1, we let

* * - *L = (T )'V (T )n _n _n_n (2.13)

where V Is defined by (1.7). Then, by an appeal to the Union-Intersection-n

principle, as in Sen (1982), or the maximin theory as in Hajek and ~id'k

*(1967), we claim that Ln is a locally maximin rank test for HO: ~ = ~,

when 0 is ignored. Let us define_n

*2 1 2 1 *2A = J {tV l (u) + tV2(u)} du = J tV (u)du, (2.14)

0 0

*2 -1 n * 2 (2.15)An = (n-1) Li =l[an (i)] •

* * *Note that A < ~ and An ~ A as n ~~. Thus, when the ~i satisfy the

(generalized) Noether condition:

1ma<i<xn (Zi-Z )'V-(Zi-Z ) ~ 0 (a.s.), as n ~~,_ "",n ......n _ "'un (2.16)

-8-

then. by an appeal to the permutational central limit theorem and the

Cochran theorem. we conclude that under HO: B- ~.

as n -+ 00 • (2.17)

2where X has the (central) chi square distribution with p degrees ofp

freedom. In particualr. for p = 1. U is a scalar (nonnegative) quantity.n

and under HO'

*-L-~ *A 11 T ~ N(O.I)n n n

We may note that by (2.5) and (2.6).

1f ~I(u)~2(u)du • -f {log Fo(x)}n(x)dSo(x)a .

• -f {log FO(x)}hO(x)FO(x)G(x)dx

= ~J FO(x)G(x)d(-log FO(x»2

2 1 2= -~ fe-log FO(x» dSO(x) = -~ f ~l(u)du

oHence. by (2.14) and (2.19)

(2.18)

(2.19)

(2.20)

Consider now a sequence {K } of alternative hypotheses. where forn

each n.

K : (1.10) holds with a = n-~An

for some (fixed) A E Eb • and assume that

-1n V -+ v (a.s.). as n -+ 00 •-n

where ~ is posotive definite (p.d.). Then. in (2.2). writing

(2.21)

(2.22)

-9-

exp(~'~i) • 1 + n-~(~'~i) + ~n-l(~'~i)2 + 0(n-3

/2). we obtain by some

routine steps tha~

(2.23)

where. under HO: 8 = O. the first term on the right hand side of (2.23)

*2 2is asymptotically normal with 0 mean and variance A (~'~~) [= 0 (say)].

the second term converges in probability to 0, while the third term to I(A'vA)*2 2A (-10). Thus, the left hand side is asymptotically normal with mean

_~02 and variance 02.. This, according to LeCam's first Lemma [viz.,

Hajek and ~id~k (1967, p. 204)],establishes the contiguity of the sequence

of probability measures under {Knl to that under HO• As such, using

*LeCam's third lemma along with the usual projection of T , it follows...n

that under {K } ,n

-~ * *2 *2n ~ ~ N(VAA , A v)_n -_ _ (2.24)

* *2Thus, under {K l, L fA has asymptotically a noncentral chi-squaren n n

distribution with p degrees of freedom (D.F.) and noncentrality parameter

* *2 1 21::.= A (~,~~) ... (Jo1jJ2(U)dU)(~'~~) • (2.25)

*Note that, in practice, to use the statistic L , one needs ton

*know the score function 1jJ • which [by(2.5)-(2.6)] depends on the unknown

FO and G. *Thus, in general, L is not an adaptable test statistic.n

Nevertheless, the above result provides a convenient means for studying

the asymptotic efficiency of other tests, and this will be taken up in

the next section.

-10-

3. LOCAL OPTIMALITY OF THE LOG-RANK TEST

In this section, we shall study the asymptotic efficie~cy and

optimality properties of the log-rank test. By virtue of the contiguity

results of Section 2 and the usual projection results on TO, parallel-n

to (2.24), we obtain that under {Kn} and the regularity conditions of

Section 2,

where

2 1 2A = J {-I - log(l-u)} du = 1

<p 0

(3.1)

(3.2)

1Y = f {-I - log(l-u)}{-~I(u) - ~2(u)}du

o1 *::: f ~ (u)log(l-u)du • (3.3)o

oThus, under {K }, L has asymptotically a noncentral chi-squaren n

distribution with p D.F. and noncentrality parameter

o 2f::" = Y (A'vA) .- -- (3.4)

By (2.25) and (3.4), the Pitman-efficiency of the log-rank test

with respect to the locally optimal one is

(3.5)

*If we write <p (u) • 10g(l-u), 0 < u < 1, then by (2.14), (2.20), (3.2)

and (3.3), we have

2 *2Y IA

1 * * 2 1 *2 2= (J <p (u)~ (u)du) I{(! ~ (u)du)A~}o 0 ~

1 * _* * 2(! (<p (u) - <p )~ (u)du)o==-----------------1 *2 1 * -* 2(! ~ (u)du)(! [<p (u) - <p ] du)o 0

2 * *== p (<P ,~ ) , (3.6)

-11-

where

2 * *p (~ ,~ ) 2 I, with the strict equality sign

* *holding only for ~ (u) ~ a~ (u) + b, 0 < u < 1,

for some real a (1 0) and b. (3.7)

2 * *By (2.5), (2.6) and (3.7), we conclude that P (~ ,~ ) = 1 only when,

log So(x) = k1 + k2 log FO(x) + k2hO(x)/{hO(x) + hG(x)} ,

(3.8)

for almost all x, and since, log So = log FO + log G, (3.8) may be

written equivalently as

log G(x) = k1 + (k2-1) log FO(X) + k2hO(x)/{hO(x) + hG(x)} •

(3.9)

for almost all x. Since hO/{hO+hG} is nonnegative and bounded between

o and 1, and log G ~ ~oo as x + +00, in (3.9), k2 has to be different from

1. (3.9) specifies the interrelationship ofFO and G for which the

log-rank test is a locally optimal rank test under the model (1.10).

An important class of distributions for which (3.9) holds may be

characterized by the two ha7.ard functions hO(x) and hG(x) as follows.

Suppose that

Then, by integration on both sides, we have

log G(x) - clog FO(x) + c', c' real,

(3.10)

(3.11)

. -1while by (3.10), hO/{hO+hG} = (l+c) • Hence, it is easy to verify

that (3.9) holds with k2 ~ c+1. Thus, if the hazard rates for the d.f.

-12-

FO

andG ~e proportional to each other, then the log-rank test is a

locally optimal rank test for the Cox model under randam censoring.

A second situation where (3.9) holds is the degenerate case where

G(x) = 1, V x < 00, so that SO(x) ... FO(X) , V x < 00, hG(x) ... 0, V x < 00,

and hence, (3.8) holds with k2""1. Thus, if the withdrawal distribution

lies completely to the right of the d.f. FO' then the log-rank test is

locally optimal as it is in the case where there is no (random) censoring.

L onn -

In passing, we may remark that Peto and Peto (1972) considered the

two-sample problem, where for some n1 (- N-n2) , Sl ...... = Sn1

... So and

Sn1+1 "" ••• "" SN = [So]l+A and showed that the locally most powerful

rank test (for A ... 0, va. A ~ 0) is the log-rank test. Their model differs

from (1.10) [in the sense that G does not remain the same under alternatives],

and hance, their conclusions may not hold for the model (1.10) even for

the special case of the two-sample problem.

So far, we have considered the local optimality and efficiency of

*the log-rank test relative to L , where the information contained in 0n ~

has not been incorporated in the testing procedures. We like to study the

loss of efficiency due to this. For this, we may note that the joint

density of the (Xi'Oi) in (1.9) is given by

n 0i 1-0i"" i~l{[fi(Xil:i)G(Xi)] [g(Xi)Fi(Xil~i)] }

so that by (1.1), (1.2) and (3.12), we have

(3.12)

0i~'~i + 0i log hO(Xi ) + (1-0i ) log hG(Xi )} •

(3.13)

-13-

Thus. under {K } in (2.21),n

1 n , 2{ - -~+ --2 Ei leA z.) log FO(Xi )} + 0 (n )n = - -~ p

(3.14)

-1 n *Thus, if n Li =l Z Z' ~ v (in probability. if the ~i are stochastic).-i-i - ~

then by (3.14). under HO: ~ = ~. as n ~ 00,

where

n = [G(x)dFO(x) = !G(X)FO(x)hO(x)dx

= !(hO(x)/{hO(x) + hG(x)})dSO(x)

= !n(x)dSO(x) •

(3.15)

(3.16)

(3.18)

and 'flex) is defined by (2.4). Now (3.15) establishes the contiguity of

the probability measure under {Kn} with respect to that under HO• Further.

by (3.13),

u = (a/aS)log L (S)I a Eni=l~i{log FO(Xi ) + oil , (3.17)-n ~ n - a=O -

. and hence, it follows by some standard steps that if In stands for the

likelihood ratio test statistic [for testing HO: ~ = ~ vs H: ~ ~ ~ on the

model (3.12)]. then under HO'

2-2 log Ln - Xp ,

while under {K }, -2 log I has asymptotically a non-central chi squaren n

distribution with pD.F. and noncentrality parameter

- *~ = n(A'v A) . (3.19)

* *Note that by (1.7), (2.21) and the definition of v • v - v is positive- -

-14-

semi-definite (of rank 1 at most), and hence

*A'V A - A'VA > 0 V A •- - - - -- - . (3.20)

[Actually, if the~i are non-stochastic, the model in (1.1) may be so

*chosen that Z = 0 and this will lead to V = V. Even, otherwise, in-0 - ~ -

(1.1), ~i may be replaced by (~i - ~n)' 1 ~ i ~ n, and this will lead to

*V = v. Thus, we may assume without any essential loss of generality that

*~ = ~ , so that the equality sign in (3.20) holds.]

By (2.25), (3.5) and (3.19)-(3.20), we obtain that the asymptotic

relative efficiency of the log-rank test relative to the likelihood ratio

test [for the model (3.12)] is

e(LO,T) = ~O/X = (~0/6*)(6*/X)

2 * * 1 2~~ (¢ ,~ ) J ~2(u)du]/TI

o2 * * J 2 J= [p (¢ ,~ )]{( TI (x)dSO(x»/ TI(x)dSO(x)}

2 * *= p (¢ ,~ )}{P2} (3.21)

2 * *where p (¢ ,~ ) < 1, with the equality sign holding under (3.7)-(3.9) and

(3.22)

with the equality sign on the right hand side holding only when TI(x) • 1

almost everywhere (SO), i.e., hG(x) = 0 almost everywhere. Note that even

if (3.9) holds [viz., (3.10)], (3.21) will be generally less than 1, due

to P2' so that there is always some inherent loss or efficiency due to

ignoring 0 and using a rank test based on the Xi alone.-n

4. ASYMPTOTIC EFFICIENCY OF THE COX PROCEDURE

If T = {t 1<•.• <tm} • {Xi (ordered): 0i=l, i=l, ••• ,n} be the set

of failure points (for which Wi exceeds Yi ), then a partial likelihood

function may be defined as in Cox (1972, 1975) as follows.

-15-

At time t.-O,J

there is a risk set Rj

of rj

individuals which have neither failed nor

dropped out by that time, for j.1 •...•mj

note that Rmc ••• cRl. Considering

the risk set Rj

and the conditional probability of a failure at time tj

(1 2 j 2 m), we obtain the partial log-likelihood function (from (1.11»

** mlog L = Ej l{S'Z * - log(Ei R eXP{S'zi})}m • ~ ~Q E j - -

j

(4.1)

* * *whereg • (Q1 •••••Qm) is a sub-vector of the anti-ranks. relating to

the indices of the observations corresponding to the ordered failures

(preceding withdrawals). For testing HO: ~ = 2 against ~ +2, Cox (1972)

considered the test statistic

where

L = U' J Unm ",nm-nm",nm (4.2)

U_nm (4.3)

Whenever, 7f, defined by (3.16) is > 0, under HO' L has asymptoticallymn

chi-square distribution with p D.F. Also, it follows from Sen (1981,

Sec. 4) that under {K } in (2.21), L has asymptotically a noncentraln nm

chi-square distribution with p D.F. and noncentrality parameter

f::, = 7f(A'VA) (4.5)

By virtue of the remarks made after (3.20), (4.4) is quite comparable to

(3.19). and this reveals the asymptotic optimality of L ,under (2.21).nm

We may note that the information on 0 is incorporated in the Cox_n

procedure through the construction of the risk sets Rj

• 1 ~ j ~ m, and

this explains the better efficiency when the model in (1.1) holds. Finally.

unlike the log-rank test, L is not a rank statistic.nm

-16-

5. SOME GENERAL REMARKS

rank

The results in Section 3 reveal the loss in efficiency of the log-

L* .test (or ) [due to ignoring the information in 0 and restrictingn -n

to the ranks of the Xi] when the Cox model in (1.1) holds; the Cox

procedure remains asymptotically locally optimal for the same model and

it incorporates the information in O. However, it may be remarked that-n

whenever under the null hypothesis, X1""'Xn are i.i.d.r.v., the 10g­

rank test is a genuinely distribution-free test. This is particularly

true when instead of the Cox model in (1.1), the Fi(xl~i) are given by

the conventional regression model F(x-~'~i)' so that Si(x) = G(x)F(x-~'~i)'

1 < i ~ n, where under HO: S • 0, Sl""'S are the same. In such a case,- - n

the log-rank test remains valid for a general class of F, G. On the

other hand the rationality and/or optimality of L for this conventionalnm

regression model may be open to questions. Thus, if we have random

censoring, it may be a basic issue whether to stick to the Cox model and

adapt the locally optimal Cox procedure (which may not be robust for

departures from the Cox model) or to use the log-rank test which remains

valid (under more general setups) and reasonably efficient for a broad

class of models. In any case, if ~, defined by (3.16) is small, ~he

(random) censoring results in substantial loss of efficiency, and, by

(3.22), a greater loss is incurred for the log-rank procedure. Hence,

when ~ is close to O,rank procedures may not be recommended.

ACKNOWLEDGEMENT

This work was partially supported by the National Heart, Lung and

Blood Institute, Contract NIH-NHLBI-71-2243-L from the National Institutes

of Health.

"

•

-17-

BIBLIOGRAPHY

Cox, D.R. (1972). Regression models and life tables. Jour. Roy. Statist.Soc. Se~. B. 34, 187-220.

Cox, D.R. (1975). Partial likelihoods. Biomet~ika 62, 269-276.

Hajek, J. & ~idak, z. (1967). Theo~y of Rank Tests. New York: AcademicPress.

Lehmann, E.L. (1953). The power of rank tests. Ann. Math. Statist. 24,23-43.

Peto, R. (1972). Rank tests for maximal power against Lehmann-typealternatives. Biometrika 59, 472-474.

Peto, R. & Peto, J. (1972). Asymptotically efficient rank invarianttest procedures. Jour. Roy. Statist. Soa. Se~. A. 34, 185-207.

Sen. P.K. (1981). The Cox regression model, invariance principles forsome induced quantile processes and some repeated significancetests. Ann. Statist. 9, 109-121.

Sen, P.K. (1982). The UI-princip1e and LMP rank tests. In CoZloquiumon Nonpa~amet~ia StatistiaaZ Infe~enae (ed: B.V. Gnedenko et.al.).Amsterdam: North Holland (in press) •