probability theory 2011 the multivariate normal distribution characterizing properties of the...

Probability theory 2011

The multivariate normal distribution

Characterizing properties of the univariate normal distribution

Different definitions of normal random vectors

Conditional distributions

Independence

Cochran’s theorem


The univariate normal distribution- defining properties

A distribution is normal if and only if it has the probability density

where R and > 0.

A distribution is normal if and only if the sample mean

and the sample variance

are independent for all n.

)2

)(exp(

2

1)(

2

2

x

xf X

n

iiX

nX

1

1

n

ii XX

ns

1

22 )(1

1



Suppose that X1 and X2 are independent of each other, and that the same is true for the pair

where no coefficient vanishes. Then all four variables are normal.

Corollary: A two-dimensional random vector that preserves independence under rotation must be normal

2

1

2221

1211

2

1

X

X

aa

aa

Y

Y

x1

x2



Let F be a class of distributions such that

X F a + bX F

Can F be comprised of distributions other than the normal distributions?

cf. Cauchy distributions



bbmaCY

eee

bteeEeeEeEt

bXaY

eeEt

xmx

xf

mCX

tbtbmaibtimbtita

XitaitbXitabXaititY

Y

timtitXX

X

,

Put

;1

,

)(

22


The multivariate normal distribution- a first definition

A random vector is normal if and only if every linear combination of its components is normal

Immediate consequences:

Every component is normal

The sum of all components is normal

Every marginal distribution is normal

Vectors in which the components are independent normal random variables are normal

Linear transformations of normal random vectors give rise to new normal vectors


The multivariate normal distribution- a first definition

Every component is normal

The sum of all components is normal

Every marginal distribution is normal

Vectors in which the components are independent normal random variables are normal

Linear transformations of normal random vectors give rise to new normal vectors

X

0,,0,1,0,,0elementth

i

iX

X1X '

11,,1 n

n

iX

normal is ,, ''

,,,,,,

121

21

kk

k

iiiiiiii

ii XXXa

X1

normal is ,,

,, i.i.d. ,,'

1

1

22

11

21

n

n

ii

n

ii

n

iiiin

XX

aaNXaNXX


Illustrations of independent and dependent normal distributions


Parameterization of the multivariatenormal distribution

Is a multivariate normal distribution uniquely determined by the vector of expected values and the covariance matrix?

Is there a multivariate normal distribution for any covariance matrix?


Fundamental properties of covariance matrices

Let be a covariance matrix of a random vector X

Then is symmetric

Moreover, is nonnegative-definite, i.e.

0

.

.

.

...,,

1

1

j

jj

p

p XaVar

a

a

aa Λ


Factorization of covariance matrices

Let be a covariance matrix.

Because is symmetric there exists an orthogonal matrix C

(C’C = C C’ = I) such that

C’ C = D and = CD C’

where D is a diagonal matrix.

Beacuse is also nonnegative-definite, the diagonal elements of D must be non-negative. Consequently, there exists a symmetric matrix B such that

B B =

B is often called the square root of


Construction of a random vector with a given covariance matrix

Let be a covariance matrix.

Derive a matrix B such that

B B’ =

If X has independent components with variance 1,

then

Y = BX

has covariance matrix B B’ =


The multivariate normal distribution- a second definition

A random vector is normal if and only if it has a characteristic function of the form

where is a nonnegative-definite, symmetric matrix and is a vector of constants

Proof of the equivalence of definition I and II:

Let XN( , ) according to definition I, and set Z = t’X. Then E(Z) = t’u and Var(Z) = t’ t, and Z(1) gives the desired expression.

Let XN( , ) according to definition II. Then we can derive the characteristic function of any linear combination of its components and show that it is normally distributed.

)2

exp()()(tΛt'

μt'Xt'X ieEt i


The multivariate normal distribution- a third definition

Let Y be normal with independent standard normal components and set

Then

provided that the determinant is non-zero.

μYΛX 1/2

)(-1/2 μXΛY

2

)()'(exp

det

1

2

1)(

12/μxΛμx

ΛxX

n

f


The multivariate normal distribution- a fourth definition

Let Y be normal with independent standard normal components and set

Then X is said to be a normal random vector.

μAYX


The multivariate normal distribution- conditional distributions

All conditional distributions in a multivariate normal vector

are normal

The conditional distribution of each component is equal to that of a linear combination of the other components plus a random error


The multivariate normal distribution- conditional distributions and optimal predictors

For any random vector X it is known that E(Xn | X1, …, Xn-1) is an optimal predictor of Xn based on X1, …, Xn-1 and that

Xn = E(Xn | X1, …, Xn-1) +

where is uncorrelated to the conditional expectation.

For normal random vectors X, the optimal predictor E(Xn | X1, …, Xn-1) is a linear expression in X1, …, Xn-1


The multivariate normal distribution- calculation of conditional distributions

Let XN (0 , ) where

Determine the conditional distribution of X3 given X1 and X2

401

062

121

Set Z = a X1 + bX2 + c

Minimize the variance of the prediction error Z - X3


The multivariate normal vector- uncorrelated and independent components

The components of a normal random vector are independent if and only if they are uncorrelated


The multivariate normal distribution- orthogonal transformations

Let X be a normal random vector with independent standard normal components, and let C be an orthogonal matrix.

Then

Y = CX

has independent, standard normal components


Quadratic forms of the components of a multivariate normal distribution – one-way analysis

of variance

Let Xijij, i = 1, …, k, j = 1, …, ni , be k samples of observations. Then, the total variation in the X-values can be decomposed as follows:

ji

k

i

k

i

n

jiijiiij

i

XXXXnXnX, 1 1 1

2.

2...

2..

2 )()(

XXXXXXXX 321 '''' AAAI


nn

nn

A

/1.../1

..

..

..

/1.../1

1

kkk

kk

kk

kkk

nnn

nn

nn

nnn

nnn

n

nn

nnn

A

11

1..

1

111.

..

.1

11

1..

111

.

.

.

.

.

11

1..

1

1.

.1

11

1..

111

111

1

11

111

3


Decomposition theorem for nonnegative-definite quadratic forms

Let

where

Then there exists an orthogonal matrix C such that with x = Cy

(y = C’x)

n

ipi QQx

11

2 ...

p

ii

p

iiii nrARankAQ

11

)( and forms quadratic negative-non arexx'

221

2211

...

.

.

...1

pp rrnp

r

yyQ

yyQ


Decomposition theorem for nonnegative-definite quadratic forms (Cochran’s theorem)

Let X1, …, Xn be independent and N(0; 2) and suppose that

where

Then there exists an orthogonal matrix C such that with X = CY (Y = C’X)

Furthermore, Q1, …, Qp are independent and 22-distrubuted with r1, …rp degrees of freedom

n

ipi QQX

11

2 ...

p

ii

p

iiii nrARankAQ

11

)( and forms quadratic enonnegativ areXX'

221

2211

...

.

.

...1

pp rrnp

r

YYQ

YYQ


Quadratic forms of the components of a multivariate normal distribution – one-way analysis

of variance

Let Xijij, i = 1, …, k, j = 1, …, ni , be independent and N( ,2). Then,

the total sum of squares can be decomposed into three quadratic forms

which are independent and 22-distrubuted with 1, k-1, and n-k degrees of freedom

3211 1 1

2.

2...

2.. )()( QQQXXXXnXn

k

i

k

i

n

jiijii

i

ji

ijX,

2


Exercises: Chapter V

5.2, 5.3, 5.7, 5.16, 5.25, 5.30

probability theory 2011 the multivariate normal distribution characterizing properties of the...

Documents