o p t i m u m experimental design for a regression on hypercube generalization of hoels result by e...

8/2/2019 O P T I M U M Experimental DESIGN for a Regression on Hypercube Generalization of Hoels Result by e Rafajlowicz

1/7

Ann. Inst. Statist. Math.Vo l. 40, No. 4, 821 827 (1988)

AOPTIMUM EXPERIMENTAL DESIGN FOR

REGRESSION ON A HYPERCUBE-GENERALIZATIONOF HOEL'S RESULT

EWARYST RAFAJLOWlCZ 1 AND WOJCIECH MYS ZKA 2l nstitute of Engineering Cybernetics, Technical University of Wrodaw,

Wybrze2e St. Wyspiatlskiego 27, 50-370 Wrodaw, Poland21nstitute of Material Science and Mechanical Engineering, Technical University of Wrodaw,Wybrzee St. Wyspiatiskiego 27, 50-370 Wrodaw, Poland

(Received S eptem ber 24, 1986; revised Septe mber 16, 1987)

Abstract. In th e note Hoel's result (1965, Ann. Math. Statbt., 36, 1097-1106) is generalized to a large family of experimental design optimalitycriterions. Sufficient conditio ns for op timal ity criterio n are given, whichensure existence of the optim um experimental design measure which is aprodu ct o f design measures on lower dimensional domains.

Key words and phrases: Experimental design, regression, D-, L-, A-,Q-optimality.

1. Introduct ionIn 1965 Ho el pub lished the fol lowing result (see Hoe l (1965a)): I f the

regression functio n

(1.1) EY(x,y) = Z ~. c~g~(x)h~(y)a=l ,8=1is est imated from uncorrelated observations in [0, 1]2, then there exists aG-opt imum exper imenta l des ign, which i s a "product" of G-opt imumdesigns for est im ating the regression functions:

s(1.2) EG(x) = ~=,a~g.(x), rEH(y) = E=Ib~h~(y).

The def ini t ion of G-optimali ty is recal led in Section 2 for readers ' conve-nience (see also Kiefer (1974), Silvey (1980), Fedorov (1982) and Pazman(1986)). The above result holds if exp erime ntal designs are al l probabil i ty

821


2/7

822 EWARYST RAFAJ]2OWICZ AND WOJCIEC H MYSZKA

measures on [0 , 1] 2 an d [0, 1], respectively, and by product of designs wemean p roduct o f the co r respond ing m easures .

Appl icat ions of th is resul t to analyt ical const ruct ion of G, D-opt imumdesigns are given in Hoel (1965a, 1965b) and Fedorov (1982). It seems,however , tha t the advan tages o f Hoel ' s decompos i t ion in cons t ruc t ingnumerical a lgori thms have not been ful ly exploi ted . As is known (see e .g . ,At wo od (1978), Wu (1978), W u and W yn n (1978), Si lvey (1980) andFedorov (1982)) , the most d i ff icul t and t ime consuming s tep in a typicali terat ive algori thm is that of choosing the next point to be included in to ades ign. Real izat ion of th is s tep , in turn , requires i terat ive search of ex-t remum of a cer tain mul t ivar iable funct ion. Diff icul t ies in i terat ive searchcan be reduced when Hoel ' s decompos i t ion can be app l i ed to a p rob lem a thand. The above facts mot ivated us to t ry to extend Hoel ' s resul t to alarger class of opt imal i ty cr i ter ions , leaving detai ls of computat ional a lgo-r i thms outs ide the scope of th is note . In v iew of recent advances inexp erim ent design theo ry (Whitt le (1973), K iefer (1974), A sh and H ed aya t(1978), Pa zm an (1980), Si lvey (1980) and Fe doro v (1982)fo r exa mple) , o urtask is not too di ff icul t , but the above s tated pract ical mot ivat ions seem tojus t ify ou r algebraic considerat ions .

We s t ress that our resul ts are val id for the, so cal led , asymptot icaldesigns (see Pazman (1986), p. 17) approximating fixed-size designs verywel l , when the number of observat ions tends to inf in i ty . This class ofexper imen ta l des igns i s commonly used in recen t monographs and con t r i -butio ns (W hitt le (1973), Kiefe r (1974), Ash and H ed ay at (1978), Atw oo d(1978), Wu (1978), Wu and Wynn (1978), Pazman (1980, 1986), Silvey(1980) and Fedorov (1982)).2. Statement of the problem and the main result

We consider the regress ion funct ion E Y ( x ) - - a t . f ( x ) of a specials t ructure, which general izes (1 .1) . Namely, the column vector f ( x ) ofcont inuous and l inearly independent funct ions on a compact set X is of theform:(2.1) f ( x ) = gl (x (1)) @ gz(x (2)) @ -- " @ gr(X It)) A= ~ g~(x(i))i= Iwh ere x t;I, i = 1,2,. . . , r a re subve ctors o f the ve ctor x. F urt her mo re, x Ill ~ Xi,i = 1, 2,. . . , r , w here Xi is a co m pa ct set an d X = X1 x X2 x ... x -Yr. In (2.1),gi" Xi ---" R m' is mi X 1 vector of con t inuous and l inearly independ ent func-t ions , whi le @ denotes Kron ecker ' s , or d i rect, prod uct of matr ices (seeMarcus and Mine (1964) , Lankas te r (1969) o r Graham (1981) fo r i t sdefin i tion) . The vector of un kn ow n p aram eters a i s an m g IrI mg colum ni=1


3/7

EXPER IMENT AL DESIGN FOR HOEL'S TYPE REGRES SION 823

vec to r , wh ich i s es t imated f rom uncorre la ted observa t ions o f Y(x0 , Y(x2), .. .wi th var iance s a 2 ind epe nde n t o f x ~ X. Ou r a im is to choo se an exper i -me n ta l des ign ~ , wh ich i s f ro m the c las s E o f a ll p robab i l i ty m easure s on Xinc lud ing a l l d i sc re te measures . Th is cho ice i s based on the in fo rmat ionm a t r i x(2.2) M(~) = fx f(x ) fr(x) ~(dx) ,as well as on an op t im al i ty c r i t e r ion O(M (~) ; m) , wh ich i s a real va luedfu n c t i o n o f t h e i n fo rm a t i o n m a t r i x M, an d i ts d i m en s i o n m (s ee W h i tt l e(1973), Kiefer (1974), Pazman (1980) and Si lvey (1980) for d iscuss ion).Every des ign ~ e ~ , wh ich max im izes O(M(() ;m) , over Z is called O-op t imum. For example , a des ign ~* e Z which max imizes

(M (~ ) ; m ) = - o "2 m a x f r ( x ) M - ' ( { ) f(x)xEXover ~ ~ ~ wi th M(~) non s ing u la r is ca l led G -op t im um exper im en ta l des ign(see Hoel (1965a), Kiefer (1974), Ash and Hedayat (1978), Pazman (1980),S i l v ey (1 9 8 0 ) an d F ed o ro v (1 9 8 2 ) ) fo r i t s i n t e rp re t a t i o n an d fu r t h e rexamples ) .

C o n ce rn i n g w e ad o p t t h e fo l lo w i n g a s s u m p t i o n s :(A.1) O is con cav e and d ifferen t iable in M, wh ich is the subset of

no ns in gu la r m atric es in M __a {M(~): ~ e ~}.Exam ples o f c r it e r ions , fo r wh ich (A.1) ho lds a re g iven in Kiefe r(1974). Let F(M;m) be an m x m m at r ix wi th e lemen ts 80(M;m)/amgj,w h ere mij are ele men ts of M . Let ~g with elem ents ~i, q i ,. ., be the class o f a l lp robab i l i ty measures on Xi. N o t e t h a t p ro d u c t m eas u re s(2.3) r~(dx) = ~ ~i(dxl')); ~, ~ ~",., i = 1 ,2 ,. .. , r ,

f o rm a s u b s e t o f ~ ," de no t ed by ~n . Fo r ~ e ~g def ine mix mg m a t r i xM~(~i) = fxgi(xli ))gf( xli)) .~i(dx li)) an d let ~ ~ ~.~ be a m ea sur e for wh ich(2.4) m a x O(Mi(~i); mi) = ( I ) ( M i ( ~ i ) ; m,) .~i~.~,I t s ex i s tence is g u a ran t eed b y co m p ac t n es s o f X ; an d b y co n t i n u i t y o f g iand O.

No te tha t fo r every ( e ~n

(2.5) M (~ ) ~-- ~ Mi(~ i) ,i=1


4/7

824 EWARYST RAFAJ LOWICZ AND WOJCIECH MYSZKA

s ince for Kronecker 's product (see Lankaster (1969) and Graham (1981))(2.6) (A ~) B) . (C @ D) = (A . C) ~ ) (B . D ) ,where A, B, C and D are matr ices of appropriate d imensions . Our aim is toshow that Hoel 's resul t ex tends to cr i ter ions with gradient matr ices Ffulf il l ing the fo l low ing con di t ions

r(A.2) If M = 1~ M~, the ni=1

(2.7) F(M; m) = (-~ F(Mi, mi)i=1

whe re M , Mi are m x m a nd mi mg matric es, respe ctively, an d m = lZI m i.i=1(A. 3) If M ~ 114+ is an m m m at ri x, th en F(M; m) i s nonnegat ive

definite; m = 1,2, . . . .Examples of cr i ter ions , for which (A.2) , (A.3) hold are g iven in the

next sect ion . We confine our at tent ion to the so cal led regular opt imal i tycriterio ns (see Pa zm an (1980) and Silvey (1980)), wh ich assure estimabili tyof al l param eters in a .

THEOREM 2.1. Under (A.1), (A.2) and (A.3) one can fin d a ~-opti mum experimental design ~ ~ ~, which is of the fo rm

(2.8) 4 ( d x ) = 4 i ( d xi=1where ~i, i -- 1, 2,..., r are defined by (2.4).

PROOF.o p t i m u m i ff(2.9)

Reca ll Kiefer (1974) that u nde r (A.1) a design ~ ~ is q~-

m ax ~U(x, ~; m) = fx ~( x, ~; m)~(dx)~:EXw h ere ~( x, ~; m) ~= T(x) F(M (~); m) f(x ), x ~ X. I t suff ices to show that for(2.8) condition (2.9) holds. From (2.1), (2.5), (2.6), (2.7), (2.8) and (A.2) i tfo l lows that

(2.10) 7J(x, 4; m) = ~ ~i(X ( i ) , ~i ; mi)i=1where 7Ji(xu), ~i;mi) ~ gT(x ( i )F(Mi(~ i) ;mi)g i (x l i ) ) . Now, our resul t fo l lows


5/7

EXPERI MENTAL DESIGN FOR HOEL'S TYPE REGRESSION 825

fro m (2. I0) an d (2.4) since apply ing (2.9) to (2.4) we get

(2.11) m a x 7 ti (x Cil, ~i; mi ) = f x 7ti(xCi)' " - (ili; mi )i(d x )x( i ) E xj i

and sim ulta neo usly we have X = X~ x 3(2 x .-. x At. O n the oth er han d,(A.3) imp lies th at ~Ui(x ;), ~i; m3 >- 0, i = 1, 2,. .., r. Co llec ting thes e facts weconclude tha t fo r ~ '= ~ bo th the m axi mu m opera t ion in (2 .9 ) and themultiple integral can be i terated. This finishes the proof.3. Discussion

Some comments concerning (A.1) , (A.2) and (A.3) are in place. Onecan not ice that (A.1) is a s tandard assumption made in the equivalencetheo rem s in differential form (see Wh itt le (1973), Kiefer (1974) an d P az m an(1980), but not ice that we use maximizat ion ins tead of minimizat ion of q~).In these papers , one can also f ind interpretat ions and formulas for di fferen-t iat ion of cr i ter ions presented below in order to indicate that (A.2) and(A.3) h old for im porta nt classes of cr i ter ions .

(1) Fr om our theory Hoel ' s resul t fol lows, s ince for D-opt im al i ty~( M ; m) = lndet M for M ~/14+ and F(M; m) = M -~. Assumpt ion (A.3)clearly holds , whi le (A.2) fol lows fro m the relat ionships[ ]1(3.1) M -1 = I~I Mi = I~I M i ~i=1 i=1 'provide d that M~, i = 1 ,2 , . . . , r are nonsingular . P ro of of the fact that for A,B no nsi ng ula r (A @ B) -1 = A -1 @ B -l can be fo un d in La nk ast er (1969)and Graham (1981) . Now, Hoel ' s resu l t fo l lows f rom the ce lebra tedtheorem of Kiefer and Wol fowi tz (Kiefer (1974) and Fedorov (1982)) onequivalence of G- and D-op t imal i ty cr i ter ions for designs from ~.

(2) In Fed oro v (1982) the class of L-op t imal i ty cr i ter ions is defined.T h e y a re o f t he fo r m ~ ( M ; m ) = - t r [ W M - l] f o r M c M , w h e re W i s a nm x m nonne gat ive de fini te matr ix of weights . In th is case, (A.3) holds ,since F( M; m) = M -I W M -~, M ~. M. Suppose tha t W can be decom posedas follows

(3.2) w= i=lwh er e W,., i = 1, 2,.. ., r are mi m i nonnegat ive defini te matr ices . Then(A.2) holds, since by (2.6) and (3.1) we obtain:


6/7

826(3.3)

E W AR YS T R AF AJ L OW I C Z AND W OJ C I E C H M YS Z KA

M-I WM-1 = ~-~ M i -1 W iM i -1 .i=1

Fac to r iza t ion (3 .2 ) a r i s e s in a na tu ra l way in the fo l lowing spec ia l ca ses(a ) A-op t ima l i ty , where W is the m x m un i t ma t r ix ,( b) Q - o p t i m a l i t y , i n w h i c h w = f x f ( x ) . f r ( x ) d x = IZI fxgi(x ('))i=I.gT(x(i))dx(i)"

g(c) C-op t ima l i ty wi th C =f (x 0) , in which W = f( xo ). fr (x o) = ~=~gi(x~o))= r,,tl) , t2/ , Irl~ is a gi ve n po in t a t wh ic hgr(x~oil), wh ere x0 t~0 ,x0 , . . . ,x 0 j

~ ( M ; m ) = - f r ( x o ) . M -1. f( xo ) i s to be maximized .(3 ) C o n s i d e r L p- n or m c r it er io n s o f t he f o r m ~ b ( M ; m ) - - - [ ( I / m )

r M-P] l/p, M ~ M, p b e i n g p o s it i v e i n te g er . A s i s k n o w n , f o r p - - - 0 w eob ta in D -op t im a l i ty c r i t e r ion , whi le fo r p = 1, A-o p t im a l i ty . We a re in te r -e s te d m a i n l y in t h e c a se p - - - ~ , w h i c h c o r r e s p o n d s t o E - o p t i m a l i t y . F o rth is c la s s o f c r i t e r ions we have F ( M ; m ) = m - V P . t r [M-P](1-p)/P.M-(p+I) and(A.3) c lear ly holds . Also (A.2) is fu l f i l led , s ince (2 .6) and (3 .1) implyM -p = ~ Mi -p and t r [m -p] = 15I tr [Mi-P]. S u m m a r i z i n g , H o e l ' s r e s u lt c ani= 1 i= 1be ex te nde d to a la rge cla ss o f regu la r op t im a l i ty c r i t e r ions and m anye x a m p l e s o f p a r t i c u l a r m u l t i v a r i ab l e d e s i g n s c a n b e c o n s t r u c t e d f r o mk n o w n o n e - d i m e n s i o n a l e x a m p l e s .

AcknowledgementT h e a u t h o r s w o u l d l i k e t o e x p r e s s s i n c e r e t h a n k s t o t h e a n o n y m o u s

re fe ree fo r h i s comments , c l a r i fy ing the p resen ta t ion .

REFERENCESAsh, A. and Hed ayat A. (1978). An introduction to design optimality with an overview ofthe literature, Comm. Statist. A--Theory Methods, 7, 1295-1325.Atwood , C. L. (1978). Convergen t design sequences, for sufficiently regular optimalitycriteria, Ann. Statist., 4, 1124-1138.Fedorov, V. V. (1982). Theory of Optimal Experiments, Academic Press, New York.Graham, A. (1981). Kronecker Products and Matrix Calculus with Applications, Horwood,Chichester.Hoel, P. G. (1965a). Minimax designs in two dimensional regression, Ann. Math. Statist.,36, 1097-1106.Hoel, P. G. (1965b). Optimum designs for polynomial extrapolation, Ann. Math. Statist.,36, 1483-1493.Kiefer, J. (1974). General equivalence theory for optimum design (approximate theory),

Ann. Statist., 2, 849-879.Lankaster, P. (1969). Theory of Matrices, Academic Press, New York.


7/7

EXPERIMENTAL DESIGN FOR HOE L'S TYP E REGRESSION 827Marcus, M. and Minc, H. (1964). A Survey of Matrix Theory and Matrix Inequalities,

Allyn and Bacon, Boston.Pazm an, A. (1980). Some features of the optimal design theory, A Survey, Math.Operationsforsch. Statist., Ser., Statistics, 11, 415446 .Pazm an, A. (1986). Foundations of Optimum Experimental Designs, Reidel, Dordrecht.Silvey, S. D. (1980). Optimal Design, Chapman & Hall, London-New York.Whittle, P. (1973). Some general points in the theory of optimal experimental design, J.Roy . Statist. Soc. Ser. B, 35, 123-130.

Wu, C. F. (1978). Some iterative procedures for generating nonsingular optimal designs,Comm. Statist. A--Theory Methods, 7, 1399-1412.Wu, C. F. and W ynn, H . P. (197 8). The convergence of general step-length algorithms forregular optim um design criteria, Ann. Statist., 6, 1273-1285.

o p t i m u m experimental design for a regression on hypercube generalization of hoels result by e...

Documents