AAET 03 STANFORD UNIV CA DEPT OF STATISTICS FIG 12/1AA8793 SI ZES OF ORDER STATISTICAL EVENTS(U)

JUN 80 0 RUDOLPHq, J M STEELE N00014-76-C-0475UNCLASSIFIED TR-287 NL

Q

cz

SIZES OF ORDER STATISTICAL EVENTS

Daniel Rudolph and J. Michael Steele

TECHNICAL REPORT NO. 287

JUNE 3, 1980

DTICPrepared under Contract SEg.KGTS

Nooo4-76-c-475 (NRo2-267) I5 ~For the Office of Naval Research

Herbert Solomon, Project Director

Reproduction in Whole or in Part is Permittedfor any Purpose of the United States Government

Approved for public release; distribution unlimited.

IDEPARTMENT OF STATISTICS

STANFORD UNIVERSITYSTANFORD, CALIFORNIA

Sizes of Order Statistical Events

By

Daniel Rudolph and J. Michael Steele

1. Introduction

One of the key properties of independent, identically

distributed continuous random variables Y.. I < i < n, is that the

order statistical events defined by

(: Yc < Y < . <Cn)

have the same probability 1/n! for any permutation 0 : [l,n] ) [l,n].

The main objective of this paper is to determine the extent to which

this property is retained asymptotically for processes which are only

assumed to be stationary and ergodic.

To set the problem precisely, we suppose that [Xi)i is a

strictly stationary, ergodic process defined on the probability space

(0, Z, P). We will also use the representation of such a process by

Xi(t) - f(T- i+ l W), 1 < i < -, where T 2 1 is an ergodic measure

preserving transformation and f : -* R is a measurable map. To avoid

inessential messiness, we also restrict attention to processei which

satisfy the continuity property

(1.1) P(Xi-Xj) 0 , i J

Our approach to the analysis of the order statistical events

is motivated by the Shannon-McMillan-Breiman theorem, and particularly

the phrasing of that result in terms of the equipartition property

([1, p. 135], [9, p. 35 (6.3)]). Loosely speaking, that phrasing

tells one in terms of the entropy of T just how many sets of a certain

type are needed to cover most of 1.

To establish a comparable result for the order statistical

events, we let %(F) be defined for any FEa by

Qn(F) v :1 :Xo( 1 )(a)<X( 2 )( )<... <Xo (n)(), for some oEFI

Here ISI denotes the cordinality of the set S, so Qn(F) is equal ton!the least number of order statistical events

Ao - Xd(1) (1 ) < XU(2)(&) < ... < Xo(n)(W) which one needs to cover F.

The quantity of main interest is now defined for E > 0 by

;*

(1.2) Qn(E) - wil 0%(n\E)E:P(E)<E

so Qn(E) is the least number of A. which will cover a set of

probability 1 - E.

To familiarize O*(E), we note that if the fXi} 1 are i.i.d.

and satisfy (1.1), then Q is equal to the least integer greater than

(l-E)n!.

In this particular example fX )CO is a process with infinite

entropy, and Q(E) is near its a priori upper bound. One intuitively

expects that n(E) should be of smaller order than n! for processes

with finite entropy. We show more precisely that in that case (E)

is, in fact, exponentially smaller.

Theorem 1. For any stationary ergodic process xi). which has

finite entropy and stisfies P(Xi X) 0, i J, there is for any

E > 0 a sequence of positive reals pn tending to zero for which

(1.3) (E) < (n!)p , for all n > I

2

Before giving the complementing result, two comments are in

order. In the first place we note there are many processes satisfying

the hypotheses, since if f satisfies P&L : f- (O) - y) 0 for all yE R,

the condition P(Xi- Xj) - 0, i 0 J, trivially holds for and any mea-

sure preserving T. Also, transformations of finite entropy not only

abound but play key roles in such distinct subjects as statistical

mechanics and the metrical theory of diophantine approximation ((1]).

Second, we note (1.3) Is equivalent to saying n(E) = o(pnn!)

for each p > 0. The phrasing of Theorem 1 was chosen in view of the

next result which makes precise the sense in which Theorem 1 is best

possible.

Theorem 2. Zor any positive pn which tend to zero there is astationary, ergodic process (Xi1= with P(X0 X 0, 1 J J, which

has zero entropy, and which satisfies

Qn(E) > (n!)pn

for infinitely many n and any E < 1.

The preceding theorem is easily seen to be a consequence of

the next result which shows that the underlying T plays a surprisingly

small role in determining Qn(E).

Theorem 3. Given any ergodic measure preserving transformation T

on a non-atomic probability space (SI, 3, P), aod given any Pn > 0

tending to zero, there is an f : -0 [0,1] which satisfies

(1.4) P(( W:0f()-x) - 0 , YxCE[,1],

3

I II I

MG;"WC ?ADU~MffJaaed

BY

d str1hutlonandn

Dist A,.aIl aivd/or(1.5) n()> (n !)Pn it sea

_ npe

for infinitely many n and any E < 1.

In the next section we give the proof of Theorem 1 as a conse-

quence of a counting argument and the application of the Shannon-

McMillan-Breiman theorem to an appropriately chosen partition.

The proof of Theorem 3 is more subtle and makes use of a

generalization of a combinatorial structure known as de Bruijn

sequences.

Since the construction provides a technique for building copies

of a finite sequence of independent random variables inside a general

stationary process, the construction should be useful in a variety of

problems.

Finally, in the fourth section a brief speculation on the

theory of order statistical events is ventured.

2. The Upper Bound Method

For any measure preserving transformation S :0 n and any

partition P (P 1~ of A, the sets given by

n-l

(2.1) nl (w :S-J(6)EP )

for some I < i < a will be called the n-p-S name associated with the

n-P-S alias (i 19 i2 ,...,ji). If S is ergodic and has entropy

H(S) < a, the Shannon-McMillan-Breiman theorem says there is an

4

no n0 (E, a, S, p) such that for n > no there is a collection of e n

of the n-p-S names whose union has probability at least I- E.

Since P(XI-X j) - 0, 1 Oj, the disjoint sets Pc given by the

permutations of tl,2,...,k) by

Pcr " ( 1) ' Xc(2) X. - (x k)

have union fl (except for a set of measure zero). The partition

p - (P I can be related usefully to the possible orderings of (X)k ..0 i-i.

Le m.a 2.1. For any n-p- Tk -name A we have

0k(A <_(nk):/(k:) n .

Proof. First consider n-2. The 2-p-T name A has an associated

sequence ( 1 i2); and i1 determines the ordering of

V,- [X 1X(),X2(6),...,Xk(W)), while 12 determines the order of

R 2 = [Xk+l(W)'Xk+2())-)"'"X2k()" To count the possible orderings of

R U R 2* we note the set R 1 determines k + 1 intervals (--, X (1) ) ,

(X(,)'X(2)),.... (XnW ) where (X )k1 are the order statistics of

{11ik . Since there are (k) ways of putting the order statistics of2k1

X kI into the k + 1 intervals, we haveI i-k+l

Q2k(A) < (2k) . (2k)!/(k)

In general, we see for Ri - (Xjk+lXj. 2 ,....X(+l)k) and 0 < J < n

t

that U R determines (t +l)k + I intervals into which the orderJ-0 J t+ )

statistics of R t+l can be placed in (k ways. Making the

sequential choices we have

In i l . . . ... . .. . . . .. --

Qnk (A) < ( k )( k) (Ik) (nk)! (k!)~

which completes the lemma. 0

To prove Theorem 1 we need to bound the number of n-P-T k

names which are needed to cover a set of probability I - E. We first

note that any nk-T- p name is contained in some n -T - name because

for any alias (i }nk- I one hasj j-0

n-l nk-in (a:T-jk MEP k A n f: T -J j E EPI

J=0 _=0

The Shannon-McMllan-Breiman theorem applied to the ergodic transfor-

mation T with entropy H(T) < L < - says there is a collection C of

2cank of the nk-p-T names whose union contains R \E with P(E) < E for

all n > n. = n0 (E, P). By the preceding remark this also implies

there is a collection C' of 2a n k of the n-P -T k names with the same

property.

We now see

nk (n\ E) < 7.Qnk (A) < 2n(nk),(k,) -

AEC n

where the last inequality follow from Lemma 2.1. For any k we can

write m - nk + r with I < r < k one has

* Q (E * ()<2 a(n+l)k ((l k!(! -n-1QM( - Q(n+l)k ()<(nll.('

provided n > n0 .Since (n+l)k!/m! < ((n+l)k) , and k! > k e-k

_k k , wehave

(E) < m! 2 l ((n+1)k) (k/e) - k

l6

and the fixed integer k was arbitrary, so

(E) = (m)

for all p > 0. The implied constant depends not only on H(T) but on T

through the no given by the SMB theorem. As noted earlier, this last

relation is sufficient to imply Theorem 1.

3. The Lower Bound Method

The first ienmma required for Theorem 3 is the so-called strong

form of Rohlin's lemma ([8, p. 22]) which provids a systematic method

for applying combinatorial constructions to stationary processes.

Lemma 3.1. Suppose T is an ergodic, measure preserving transfor-

mation on a non-atomic probability space (SI, 5, P). For any finite

partition A - {HIH 2,...,H sI of n and for any real E > 0 and integer m

there is an EE U with the following properties

(3.1) E, T-1 E,...,T -m + l E are disjoint

m-1 -i

(3.2) P( U T E) > 1 - Ei-0

! i0

(3.3) P((T E) ) P(E)P(H) 0 < I < m , 1<j•s

The second lemma we need is a graph theoretic result due to I. J. Good

([5], [6, p. 95]) which sharpens a well-known result of Euler.

Lemma 3.2. If G is a connected directed graph, and ii at each

point of C there are the same number of arcs going out as coming in,

7

then there is a directed cycle in G that goes through every arc of G

in its given direction, and uses no arc twice.

As an application of Lemma 3.2, we will obtain the existence

of what can be called s-ary de Bruijn sequences. To introduce these

sequences, we recall the classic result of de Bruijn which says the

following:

Given a positive integer n, there is a sequence of O's

nand l's of length N=2 , say aIa 2a3 ... aN, such that the n-

tuples aia i+l ... a i+n-1 are a complete list of all 2n of the

n-tuples with alphabet'{0,1}.

Here a1 is understood to follow aN, etc. in the cycle. For example,

when n= 3, the cycle (00010111) contains every 3-tuple of O's and l's

exactly once. We will use the following generalization where a is an

alphabet of s letters and B is the set of all ordered k-tuples of G.

Lemma 3.3. There is a sequence a a aN with N= sk of the ele-

ments of G such that each element of a occurs exactly once in the set

of k-tuples (ar+l,ar+2,...,ar+k), where 0 < r < sk and at = a if

k kt > s and t-s =U.

Proof. We define a directed graph G whose edges are the ordered

(k- l)-tuples formed by elements of G. We have an edge from

(b1b 2 ... bkl) to (bl,...,bk_,) provided b2 =b" bkb,...,b-2 = bk -l

and b is arbitrary. Every vertex has in-degree and out-degree

equal to s, so Lemma 3.2 implies there is a cycle which traverses the

edges of G and uses each exactly once. From such a cycle the sequence

8

of ai E C given by the successive bk_ is easily checked to satisfy

the claim of the lemma.

The proof given of Lemma 3.3 is only a mild modification of

the application given by Good [5] and which Bondy and Murty [2,

p. 181] relate to the design of an efficient computer drum. A com-

pletely different algorithmic proof of Lemma 3.3 was developed inde-

pendently in recent work of Fredricksen and Nariorana [4].

We now proceed to prove Theorem 3 by applying the preceding

lemmas infinitely many times.

We suppose now that (pi* 1, {t l {h ) are increasing

sequences of positive integers and fE I is a sequence of positivei s=1

reals decreasing to zero. We will define a sequence of functions

gO(1) s)0, l(J), 2),. gk(&) .... where each gk(w) will be

defined via pk' tkt hk, Ek' and the preceding g (w), 0 < j < k. Also,

we should remark that each of the gi(cj) will assume only finitely many

values.

For notational convenience we temporarily write p = Pk' t = tk,h = , and E - E . We define G by fa : a = p l Er 2 - r 9 Er

= 0 or 1,

k3 k r r

1 r < p), and note by Lemma 3.3 there is a sequence of IJGt = 2tp

elements of G which form a cycle in which each ordered t-tuples with

letters from G appears precisely once among the (ai+l,ai+2 ,... ai+t)

for 0 < i < 2tp

We now apply Lemma 3.1 to obtain an E such that for

0 < k < 2h 2 tp the sets T AE are disjoint and there union has prob-

ability at least 1 - E. For the finite partion 9 we take the

9

partition given by the distinct values of the sum Zi= 1 gi()2 1 wherei=l 1

qJ PI + P2 + "" + Pj, < _j < k.

We now define gk(03) = a,, if wET -i+ l for 1 < i < h2t p and

i i' mod 2t p . Finally, we take gk(i) = 0 if W is in none of the

S-i+lE, 1 < k < h2tp.

The whole point of this construction is that now by setting

Q equal to the union of T-i E with 0 < i < (h-l)2tp and lettingk

Pk(A) = P(Anlk)/P( k), we see that the random variables gk()

gk (T-1 )I,...kg(T-t+l() are independent in the probability space

(Sk' Pk' k) where 3 k = {Anf : AE 3). More over, these rindom vari-

ables are also independent when conditioned on the a-field given by

the partition A. To prove these assertions one only has to note that

for any HE A we have by (3.1) and (3.3)

S-Pk -t+l -Pk nH)P({gk(ci) =S 2 - k gk c- ) =s-12 '''g ci)s 2 ktl2 H

k 0 -t+l

= (h -)P(E)P(H)/(h-1)2tp P(E) = 2- t p P(H)

Finally, we are able to define f(w) by letting q = kj.l

and setting

-qf(o) E . gk (wO)2

k=l

The sum representing f(w) converges for all w, and one finds no diffi-

culty in checking that for Xi(6j) = f(T-i+ic ) wL. have P(Xi=X) = 0 for

all i # J.

10

To prove Theorem 3 we suppose that the sequence p 4 0 and

C E (0,1) are given, and we will proceed to show that [p fti}) -,

I and fE ) can be chosen so that Qn(E) > n Pn for infinitely

many n.

For the intervals Is = [s 2 - qk - , (s+l)2 - qk - l) with

qk-

0 < s < 2 we define random variables V as the number of elements

of J = {i: XEI s, < i < n. The ordering of fXiE is completely-- i iEJ ometl5

determined by the ordering of {gk(T-i+l ) iE except for at most those

-i+l -J+l s(aE = {U g k(T (0) = gk(T to), for some 0 < i < j < n). Using

the Pk-independence of the {gk(T-i+lco 1 <1< i < n for n < t and the

conditional independence given 9 we have

P({X < X < ... < X IlH)<() + h(2) < .(n)

< E + h -1+ P(f{x(1 <Xd2 < ... < XF (n) fnu )-k k l a( ) r n

1

<Ek +hkl +Pk(fX (1) < Xo(2) < ... < (n H)

< Ek + hkl + Pk(3) + Pk(H)( .' Vk- k 3 k s

ns

Since there are only (2) places a tie can take place and the-qk

P k-probability of any such tie is 2 , we have

P (J)< (I- E - h kl- PkGO < 2 (2) 2 - < n 2 2 -q

k k Pk 2

Also, Pk(H) = P(H) for all HEX by (3.3); so summing over N we have

P(x0(1) <X (2) < ... < Xc()) < 2 l (Ek+hk ) + n2 2 + I (VaS

Setting r= 2q k - 1 we have Z r-I V n. Hence, (vs') =s=o s S

Tir (v +1) > r-) by the convexity of logr (x), [7, p. 285]. Wes r

8begin with the bounds

(3.4) qn(E)>(l-E)fmax P (XC1) < X (2) < < -

n-Irr +-k 2-

> (1-E)n!r(n+l)r(---) +r(n+l)(2 n 2+rEk+rhk)}1- .

Since r = 2 is fixed (as we choose Ek2 Pk' tk" and hk), we invoke

Stirling's formula to obtain r(n+1)r((n+r)/r) - r < (r+l)n for

n > n0 (r). Since pn ' 0, we can now choose t tk _> no so that

(3.5) (r+l)-t > 2pt(l-E) -ItFinally, we choose Ek, h and Pk so that

-P k 2 kt(3.6) {r(t+l)( 2 t2+rEk+rh) > 2 (IE) -1

rkrk Pt

By the elementary inequality l/(a+b) > (l/2)minfl/a, 1/b) applied to

(3.4), (3.5), and (3.6) for each t = tk, k=l,2,... we have

*%(E) > P(n!) for n -t,t 2 . . .. . .

The proof of Theorem 3 is thus complete. 0

4. A Brief Speculation

The only two cases where one has a reas6nably complete under-

standing of Q*(E) are the case of continuous i.i.d. random variables

and now in case of continuous stationary ergodic processes with

12

finite entropy. These two cases are in a way polar opposites of

generality, and many important classes of processes lie in between.

Since many basic probabilistic events are simply unions of the

order statistical events, it would seem to be of considerable interest

to discover those cases in which a precise understanding of Qn(E) can

be obtained.

13

References

[1] Billingsley, P., Ergodic Theory and Information, John Wiley andSons, New York, 1965.

[2] Bondy, J.A. and Murty, U.S.R., Graph Theory With Applications,American Elsevier Publishing, New York, 1976.

[3] de Bruijn, N.G., "A combinatorial problem," Nederl. Akad.

Wetensch., Proc. 49 (1946), 758-764; Indagationes Math, 8(1946), 461-467.

[4] Fredricksen, H. and Maiorana, J., "Necklaces of beads in kcolors and k-ary de Bruijn sequences," Discrete Mathematics23 (1978), 207-210.

[5] Good, I.J., "Normal recurring decimals," J. London Math. Soc. 21(1946), 167-169.

[6] Hall, M., Jr., Combinatorial Theory, Blaisdell Publishing,Waltham, Mass., 1967.

[7] Mitrinovi6, D.S., Analytic Inequalities, Springer-Verlag,New York, 1970.

[8] Shields, P., The Theory of Bernoulli Shifts, University ofChicago Press, Chicago, 1973.

[9] Smorodinsky, M., Ergodic Theory, Entropy, Lecture Notes inMathematics, Vol. 214, Springer-Verlag, New York, 1971.

I1

14

SCCUITY CLA11FCAT10% 0, TinS PO-GE MWO a. meto ________________

REPORT DOCUMENTATION PAGE NA ITUm

1.M N111x-fu81 GOVT ACCESGO 100 1119CUPICAYVS CAVALOG OUM~

287 _ _ _ _ _ _ _ _ _ _ _ _ _ _

( SJ IZES OF_9JWUERTATISTICALEVENTS TEHIAP

7.A6.NW# ,EfN W pEe- -wap

W AIEL */RDOLPH 4 J. MICHAEL STEELE

Department of Statistics ~"'Stanford University no2.6

Stanford, CA 9I.305I.CONTROL- %z " c 9'i~ NA049 "o0 00"as5 -

Office of Naval ResearchStatistics and Probability Program Cod- 'a. !!-n om o , sArJinaton. Virainia, 22217 14___________

14. WMNITO AN .8~ IN A A413S(II dBifft"O Si.. COad.MP98 *OR&*) 'IL -SwcuRrY CLASo. (.1 ai. -0

160 DCLASSIFICIlfDOWGA10scCOULA

APPRC'-:7E Fp?' PUBLIC RELEASE: DISTRIBUTION UNLIMITED.

17. DISTRIU3 : A-EW1147 :at MI. abUC1s U,1.,j in 8109k 20, It 41100 1 .ow Aftl...j

Is. SUPPLEM!t--m '40TES

Or satitc;En;top;:aioyprcesse; de Br" ijn sequences;

20. AfsS' RACT fCwI.Mw& e pa"* @#I*. 1 It no~o...p &* #Idtfr by bjecir m.'

PLEASE SEE REVERSE SIDE.

DD , 1473 tDITION OP is NOV sot@68LET

1'N~~Rr 0)2 ~IF4?c6ASUIICATIOW OV ?NijS 10,o F==0 BMW-*

MCLASSIFIED

SECURITY CLASIFICATION OF THIS PAGE olm a Es dW**

#287

SIZES OF ORDER STATISTICAL EVENTS

Given a process (Xl1, any permutation a : [1,n] * (1,n]

determines an order statistical event A(C) - (X( 1 )< X (2)< ... Xd~l) (2)0(n)

now many events A(a) are needed to form a union whose probability

exceeds 1-E? This question is answered in the case of stationary

ergodic processes with finite entropy.

UCAIII/N 0@5.- Old 04- 51

UNCLASSIFIED.WPilm.iV. ae qeiWiPa~,#lu AU Yui@ A~PIpeI R.. 9...dih