This article was downloaded by: [Southern Illinois University]On: 18 December 2014, At: 11:15Publisher: Taylor & FrancisInforma Ltd Registered in England and Wales Registered Number:1072954 Registered office: Mortimer House, 37-41 Mortimer Street,London W1T 3JH, UK

Communications in Statistics- Theory and MethodsPublication details, including instructions forauthors and subscription information:http://www.tandfonline.com/loi/lsta20

Regular mles for nonregulardistributionsShaul K. Bar-Lev a & Camil Fuchs aa department of Statistics , University ofHaifa , Haifa , 31905 , Israelb Department of Statistics and OperationsResearch Tel , Aviv University , Tel Aviv ,69978 , IsraelPublished online: 27 Jun 2007.

To cite this article: Shaul K. Bar-Lev & Camil Fuchs (1999) Regular mles fornonregular distributions, Communications in Statistics - Theory and Methods,28:9, 2037-2044, DOI: 10.1080/03610929908832404

To link to this article: http://dx.doi.org/10.1080/03610929908832404

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of allthe information (the “Content”) contained in the publications on ourplatform. However, Taylor & Francis, our agents, and our licensorsmake no representations or warranties whatsoever as to the accuracy,completeness, or suitability for any purpose of the Content. Anyopinions and views expressed in this publication are the opinions andviews of the authors, and are not the views of or endorsed by Taylor& Francis. The accuracy of the Content should not be relied upon andshould be independently verified with primary sources of information.

Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly inconnection with, in relation to or arising out of the use of the Content.

This article may be used for research, teaching, and private studypurposes. Any substantial or systematic reproduction, redistribution,reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of accessand use can be found at http://www.tandfonline.com/page/terms-and-conditions

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14

COMMUN. STATIST.-THEORY METH., 28(9), 2037-2044 (1999)

Sh~u! K. Ear-Lev' and Camil Fuchs2

'Department of Statistics, University of Haifa! Haifa 31905, Israel

2Department of Statistics and Operations Research Tel Aviv University, Tel Aviv 69978, Israel

Kcg CV"rds. maximum likeiihoad est:mates: GkeIihced function . - ~

Distributions whcse extremity vahes of the support depend on unknown pa- rameters are usually known as nonregular distributions. i n most cases, the MLEs for these parameters cannot be obtained by differentlation. Familiar examples are the uniform distribution on the interval (O,9) and the truncated exponential distribution with truncation parameter 0. However, there exist distributions whose extremity points of the support depend on unknown pa- rameters, which nevertheless are reguiar in the sense that the X i E s can be obsaincd bv diEere::tiatien. This note provides a method of constructing snch nonregular distributions with regular MLEs.

I . INTRODUCTION

The problem of searching for the maximum likelihood estimates (MLEs) is usually divided into two main types. The first type deals with the regular case in which the likelihood function (LF) is twice differentiable and the MLE is the solution of the likelihood equations. The procedure in this case is to find the stationary points of the first derivative of the LF and to determine the global maximum by examination of the second derivatives.

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14

2038 BAR-LEV AND FUCHS

The second type is related to nonregular distributions in the sense that one or two of the extremity points of the support are unknown parameters. A rather familiar example of this type is the Lr(O, 6) distribution. The support of the distribution is [O, 61 and the parameter space is (0, cm). Based on a random sample of n observations ( r l , 22,. . . ,I,), the positivity domain of the LF is :z,,~, x), K h ~ i ~ z,,) = max(z,, r * , . . . , z,), aiid its iiiifity daiiiak is (0, a(,) j. The MLE for 6 is thus obtained at z(,); an extremity point of the positivity domain of the LF, where the LF is not continuous and therefore not differen- t~able. 'I'he standard techniques for finding the maxima using differentiation are thus not applicable in this case.

Another familiar example of the second type is he exponentid distribu- tion with unknown location parameter 6. For this distribution, the support is (-cm, 6Jj and the parameter space is W. Based on a random sample of size n, thr LF is positive over (-m, x!~!], where xf l ! = min(zl.. . . . x,) and vanishes elsewhere. The MLE is attained at x(,!> again an extremity point of the pos- i!lvit;- :!-:nain of the LF, ::-here the LF is i i ~ t continuous and thereffire nrji . - <- . , .7>mG?G,?r:>,-.,G .'.,,. ,. .,. ,..., ..

77 nnih i 'ne ;Tjfi:i?) ilnri thr= trunrat~d pupnn~nt;;ri riistrihwicE z e ~ ~ ~ r r 6

dar with t lx t reu~i l~ p u i ~ ~ l s ur I l ~ e support depending on unknown parameters, and have nonrcgular MLEs in the sense that they cannot be found Iry differen- tiation. However; there are certainly many nnnregular distribntions ir! which the MLE is regular and can be found by differentiation at an inner point of the positivity domain of the LF.

Interestingly enough, the textbooks of statistical inference contain only very few examples of this type. One such example can be found in Freund and Walpole (1989), where the following example is given as a problem: Let x be an observation taken from a distribution with density

What is the MLE? In the example presented in this problem, the support is [O,0], 0 E (0, oo), and the positivity domain of the LF is [x, oo). An extreme point of the support is thus 6 and an extremity of the LF is z. But in this case, the LF is continuous in 6, vanishes at 6 = z and is maximized at 6 = 3/22 > x, where the LF is twice differentiable. Thus, the MLE is obtained by the standard method of differentiation.

This note is aimed at providing a method to construct a large collect~on of examples of this type, i.e. distributions in which although the extreme point of the support is the unknown parameter and the extremity of the positivity

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14

REGULAR MLEs FOR NONREGULAR DISTRIBUTIONS 2039

dnmain is a function of the observations, the MLE is regular in the sense that it is obtained by standard differentiation a t an inner point of the positivity domain of the LF.

For simplicity and ease of presentation, we confine ourselves in this sequel to the one observanon case. We address the case with n observat~ons in Section 3

2. THE METHOD OF CONSTRUCTION

The method of construction of the proposed collection of examples is based on a closer look at the density presented in (1). This density is actual11 the density of the second order statistic of a sample of size 3 from a U(0,B) distribution.

More speclficaily, the T-th order statistic, .i 5 r 2 3, out of three i.i.d. ~bservat ions from a U ( O , 8 ) distribution has a denslty,

. . . , . . . . . - . . - - . . where c,h = k ! / I ( r - 1 ) ! ( k - r ) ! i . Note that for r = 2 , the density in (2,) coincides with that in (1). For aii three czses T = 1.2 .3 . the s ~ p p u r i is :O. $1. and illr - - . . . rvsitii.~ty domai:: af the LF is ( z l m:) Fcr r = 1 . 2 t he LF ia cor?tir??~oxs and differentiable and the MLE is obtained at an inner point of the (x, w) interval. The MLEs are e = 3x for r = 1 and 0 = !a: for r = 2. However, for r = 3,

the LF is not continuous at x and the MLE is obtained a t e = x. Figure 1 presents the three cases with - 1 , 2 , 3

In the construction of the collection of examples similar t o those in Figure 1, we use t h e fact that both the U(O, 9) and the exponential distribution with location parameter 0, are special cases of families of density functions intro- duced by Hogg (1956) and are obtained as follows: Let (a, b), -00 5 a < b 5 co be a given interval. Let h be a positive functlon on (a, b), which is integrable on every closed interval contained in (a, b).

Denote e

m l ( 6 ) = ./ h(u)du, 9 E (a, b) b

rn2(S) = J h(u)du, 9 ; (a, 6 ) . (3)

8

Both m l and m, are strictly monotone on (a, b), where m i is strictly increasing ai,d iji2 j5 strict:y decrcasiiig, ~ ~ x ~ r ~ da,-,t- their inverses by my1 and my1, r~spectively. B y employing ml and m a given in (3) , Hogg (19.5.6) defines two - -- families of one-parameter distributions having density functions:

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14

BAR-LEV AND FUCHS

FIG. 1 The likelihood functions of 0 based on the T-th ordered statistic, for r = 1,2 ,3 , from the U(0,O) distribution, for x = 0.8.

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14

REGULAR MLEs FOR NONREGULAR DISTRIBUTIONS

f i( . . 6 ) = { ~ ( ~ ) , / ~ , ( 0 ) . elsewhere i r 5 0

and

' ; , ( , ) / j i i 2 ( 8 ) , 8 5 r : 6 f z i x : 8) =

elsewhere

It can be easliy shown that the density functions of the r-th order statistic, out of E i.i.d. ~ .Y . 'F? , stemming from (4) and (51, respectively, are

Proposition 1 Let x be a n observation taken from a parent density (6). T h e n the positivity part of the likelihood function (6) is (a, b) . If 1 5 T < k , the like- l ihood~~f - ;nc t zon is c ~ n t i n z o u ~ everywhere o n the parameter space ( a , b ) , and is twice differentzable there. Moreover, the M L E for 6 is obtained by di ferent i - s t ion at a n interior poznt of (z, 6). [ f?- = k, the I ~ I i E j o r 6 i s aiiaineci at z,

an extrem.zty point of (2, b) and a discontinuity point of the likelihood function. More specifically, the M L E for 0 for both cases is given by

wzth < x for 1 5 r < k .

Proof. The case with k = r follows immediately since the likelihood function vanishes at (a,x) and is strictly decreasing otherwise. Hence 6,,k = x . Let i <-. - T <: k, t h e n the likclihwd i'iixctior, (6) vanishes at x and is continnous on (a, b j since m l ( 0 ) is such. Hence the MLE, if it exists? is obtained at an interior point of (2, b). Accordingly, for x < 6 < b, denote g,.k[@) = log fl,,,k(x : 6 ) . Then, differentiation g with respect to 6 gives

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14

BAR-LEV AND FUCHS

k - r -") . ':*(') = h(8) (ml(6) - ml (z) ml(6) '

Since ml is monotonically strictly increasing, the only solution of (9) satisfies

where the negativity in (12) is followed by (lo), since

Hence er,k in (11) is the MLE for 8. 0 Note that the three cases of Figure 1 follow from (8) with k = 3 and

m l ( 8 ) = e. Results similar to those in Proposition 1 can be derived analogously if

x is an observation from a parent density (7). In such a case the likelihood function is positive on (a, 2) and the MLE for 0 is

where e,,t < x for 1 < T k

3. EXTENSIONS

The examples constructed in the previous section were confined to a single -L ---..- A:--&? L _ - L C_-I1. T l I 1 1 . 1 1.

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14

REGULAR MLEs FOR NONREGULAR DISTRIBUTIONS 2043

and remark on the method of construction for both the n observations case as well as for a two-parameter family.

If the likelihood function in Proposition 1 is based on a sample of n observations taken from ( 6 ) , the positivity domain of the likelihood function is jx(,), b) . If T - k the MLE is xi,j - a discontinuity point of the likelihood function. If i 5 T i k . i~ can be seeu i h t the :ikeLXood fii-ilction is c ~ ~ t i n ~ = x everywhere on the parameter space ( a , b ) , is twice differentiable there and vanishes at z~,). The MLE will therefore be an interior ~ o i n t of (x ( , )$ bj! but not necessarily obtainable as an expLcit fmction of the observations as in (8j with n = 1, but rather is given as an implicit root of the likelihood equation.

A two-parameter type family can also be developed in a nanner similar to that in ( 4 ) or (5). Reusing the definitions of a, b and h there, define

Here. the joint density of the T-th a d S-th order statiatlcs, out of k i.i.d. r .v 's

from (13j, with i 5 T 5 Y (I k, can be showxi to have the fcrrr?

k! fr,s,k(x, $/ 01, 62) = (T - l j ! ( s - T - l ) ! (n - s ) !

If (14) is viewed as the likelihood function of (81: d 2 ) , then its positivity part is j a , x ) LJ ( y , b ) . If T = 1 and s = k the MLE for (B1,02) is obtained at the extreme points (x, y ) . If, however, 1 < T < s < k , the MLE is solved by the standard differentiation technique.

Obviously there are other examples of distributions which do not fall in the realm of the three families presented by ( 6 ) , (7) or (14) . For instance, the Weibull distribution with location parameter 6 has a density

> 1 ih hi^ &tribi;t:.-. . . ,~i - -! ,,a:,2 --- an important rolp in life testing and reliability. Based on one ohservation the likelihood function is positive on (-00, x ) and vanishes otherwise. The MLE here is not attained at x but at

= r - [ (a - l)/a]'" < x - an interior point of (-m, x ) - b y the standard

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14

2044 BAR-LEV AND FUCHS

technique of differentiation. If, however, n observations are involved, once again the MLE is not attained at q l ) , i.e., not at the right extremity point of the domain of positivity of the likelihood function but rather then on some interior point of this domain. The MLE is then given as an implicit root of the first derivative of the likelihood function (see Johnson and Kotz, 1970).

BIBLIOGRAPHY

F r ~ u n d , J E and Walpole, R. E. (1989). Mathematical Statistics (4th cd.), Prentice-Hall.

Hogg, R. V. (1956). "On the distribution of the likelihood ratio," Thc Annals of Mathematical Statistics, 27, 529-532.

Johnson, N. L. and Kotz, S. (1970). Continuous Univariate Dzstributions- 1, John Wilev.

Dow

nloa

ded

by [

Sout

hern

Illi

nois

Uni

vers

ity]

at 1

1:15

18

Dec

embe

r 20

14