The Multivariate Normal Distribution, Part 1

BMTRY 7261/10/2014

Univariate Normal• Recall the density function of the univariate normal

• We can rewrite this as

2

2

2

for , the density function is

1 1exp

22

X N

x x x

Multivariate Normal Distribution• We denote the MVN distribution as

• What is the density function of X?

~ ,pNX μ Σ

Multivariate Normal Distribution• What is the density function of X?

Multivariate Normal• Note, the density does not exist if– S is not positive definite– = 0– does not exist

• We will assume that is positive definite for most of the MVN methods we discuss

Σ1Σ

Σ

Multivariate Density Function• If we assume that S is positive definite

is the square of the generalized distance from x to m.

• Also called– Squared statistical distance of x to m.– Squared Mahalanobis distance of x to m– Squared standardized distance of x to m

1' x -μ Σ x -μ

Why Multivariate Normal• The MVN distribution makes a good choice in

statistics for several reasons– Mathematical simplicity– Multivariate central limit theorem– Many naturally occurring phenomenon approximately

exhibit this distribution

Bivariate Normal Example• Consider samples from

• Let’s write out the joint distribution of x1 and x2

2~ NX μ,Σ

12

112

1exp

2

x x -μ 'Σ x -μ

Σ

Bivariate Normal Example

• Joint distribution of x1 and x2

Bivariate Normal Example

• Joint distribution of x1 and x2

Bivariate Normal Example

• This yields joint distribution of x1 and x2 in the form

1 22

11 22 12

2 2

1 1 1 1 2 2 2 2122

1 1 2 212

1,

2 1

1exp 2

2 1

x x

x x x x

Bivariate Normal Example

• The density if a function of m1, m2, s1, s2, and r– The density is well defined if -1 < r < 1– If r = 0, then … 1 2,x x

Contours of constant density• What if we take a slice of this bivariate distribution at a

constant height?– i.e. 1' constant x -μ Σ x -μ

Contours of constant density• The density is constant for all points for which

• This is an equation for an ellipse centered at

2 1

2 2

1 1 1 1 2 2 2 2122

12 1 1 2 2

constant '

12

1

c

x x x x

x -μ Σ x -μ

1

2

x

x

x

1

2

μ

Bivariate Normal Example

• Let’s look at an example of the bivariate normal when we vary some of the parameters…

Examples

11 220, 11 220,

11 220, 11 220,

X1

X1

X1

X1

X2

X2 X2

X2

Contours of constant density• What happens when 11 220,

Contours of constant density

• How do we find the axes of the ellipse?– Axes are in the direction of the eigenvectors of S-1

– Axes lengths are proportional to the reciprocals of the square root of the eigenvalues of S-1

– We can get these from S (avoid calculating S-1)

• Let’s look at this for the bivariate case...• We must find the eigenvalues and eigenvectors for S– Eigenvalues:

– Eigenvectors:

0 I

0 I e

• Eigenvalues of S:

• Eigenvalues of S:

• The corresponding eigenvector, e1, of S:

• The corresponding eigenvector, e1, of S:

• Similarly we can find e2, which corresponds to l2 :

• The axes of the contours of constant density will have length for 1, 2,...,i ic e i p

• If we let then

• are the eigenvalues of S and e1 and e2 are the

corresponding eigenvectors1 2

2~ NX μ,Σ 2~ ,N z x μ 0 Σ

• The ratio of the lengths of the axes

• The actual lengths depend on the contour being considered.

• For the (1-a)x100% contour, the ½ lengths are given by

• Thus the solid ellipsoid of x values satisfying

has probability 1-a.

22, i ie

1

2

length of major axis

length of minor axis

1' x μ Σ x μ

• Univariate case: length of the interval containing the central 95% of the population is proportional to s

• Bivariate case: the area of the region containing 95% of the population is proportional to .1

2

1 2

• We can call this “smallest” region the central (1-a)x100% of the multivariate normal population.

• The “area” of this smallest ellipse in the 2-D case is:

• This extends to higher dimensions (think volume)– Consider– The smallest region for which there is 1-a that a randomly selected

observation falls in the region is a p-dimensional ellipsoid centered at m with volume

122

2, 1 2area constant Σ

~ pNx

2 2 22

,

2

2p p p

ppp

Σ

• Visual of the 3-dimensional case

1

2

3

x

x

x

x

Next Time

• Properties of the MVN…