more on distributions of quadratic formsdistributions of quadratic forms martin ohlson outline...
TRANSCRIPT
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
The 8th Tartu Conference on MULTIVARIATE STATISTICS
The 6th Conference on MULTIVARIATE DISTRIBUTIONS with Fixed Marginals
More on Distributions of Quadratic Forms
Martin Ohlson and Timo KoskiLinkoping University, Sweden
Tartu, EstoniaJune 29, 2007
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Outline
1 Introduction
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Outline
1 Introduction
2 Quadratic FormsUnivariate Quadratic FormsMultivariate Quadratic Forms
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Outline
1 Introduction
2 Quadratic FormsUnivariate Quadratic FormsMultivariate Quadratic Forms
3 Example
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
This talk is based on my licentiate thesis (Ohlson, 2007)
”Testing Spatial Independenceusing a
Separable Covariance Matrix”
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Testing Independence
Let the matrix X : (p × n) have matrix normal distribution witha separable covariance matrix, i.e.,
X ∼ Np,n (M, Σ, Ψ) .
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Testing Independence
Let the matrix X : (p × n) have matrix normal distribution witha separable covariance matrix, i.e.,
X ∼ Np,n (M, Σ, Ψ) .
Assume that Ψ is unknown, but has some structure.
AR(1)
Intraclass structure
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Testing Independence
Let the matrix X : (p × n) have matrix normal distribution witha separable covariance matrix, i.e.,
X ∼ Np,n (M, Σ, Ψ) .
Assume that Ψ is unknown, but has some structure.
AR(1)
Intraclass structure
My problem is to test spatial independence,
H0 : Σij = 0, when i 6= j ,
where
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
X , M and Σ are partition into k parts, as
X =
X1
...Xk
, µ =
µ1...
µk
and
Σ =
Σ11 Σ12 · · · Σ1k
Σ21 Σ22 Σ2k
......
. . .
Σk1 Σk2 Σkk
,
where Xi : (pi × n), µi : (pi × 1) and Σij : (pi × pj) fori , j = 1, . . . , k .
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Introduction
Several authors have investigated the density function for amultivariate quadratic form. The density function involves thehypergeometric function of matrix argument, which can beexpand in different ways.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Introduction
Several authors have investigated the density function for amultivariate quadratic form. The density function involves thehypergeometric function of matrix argument, which can beexpand in different ways.
Khatri (1966) - zonal polynomials
Hayakawa (1966); Shah (1970) - Laguerre polynomials
Gupta and Nagar (2000) - generalized Hayakawapolynomials
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Introduction
Several authors have also investigated under what conditionsthe distribution for a multivariate quadratic form is Wishart.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Introduction
Several authors have also investigated under what conditionsthe distribution for a multivariate quadratic form is Wishart.
Rao (1973)
XAX ′ ∼ Wp ⇔ v′XAX ′v ∼ χ2
Gupta and Nagar (2000)
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Univariate Quadratic Forms
q = X′AX,
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Univariate Quadratic Forms
q = X′AX,
where
X ∼ Np(µ, Σ).
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Univariate Quadratic Forms
q = X′AX,
where
X ∼ Np(µ, Σ).
What is the distribution of q?
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Two Theorems from (Graybill, 1976).
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Two Theorems from (Graybill, 1976).
Theorem (I)
Suppose that X ∼ Np(µ, Σ), where rank(Σ) = p and let
q = X′AX. Then the distribution of q is noncentral χ2, i.e.,
q ∼ χ2k(δ), where δ = 1
2µ′Aµ, if and only if any of the
following three conditions are satisfied
1. AΣ is an idempotent matrix of rank k.
2. ΣA is an idempotent matrix of rank k.
3. Σ is a c-inverse of A and rank(A) = k.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Theorem (II)
Suppose that X ∼ Np(µ, Σ), where rank(Σ) = p. The random
variable q = X′AX has the same distribution as the random
variable
w =n
∑
i=1
diwi ,
where di are the latent roots of the matrix AΣ, and where wi
are independent noncentral χ2 random variables, each with one
degree of freedom.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
A generalization of the second Theorem from(Graybill, 1976) for the multivariate case can be found
in (Mathai and Provost, 1992).
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
A generalization of the second Theorem from(Graybill, 1976) for the multivariate case can be found
in (Mathai and Provost, 1992).
In this talk we will also generalize the second Theorem, but ina slightly different way using the characteristic function.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Multivariate Quadratic Forms
Q = XAX ′,
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Multivariate Quadratic Forms
Q = XAX ′,
where X is a normally distributed matrix.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Multivariate Quadratic Forms
Q = XAX ′,
where X is a normally distributed matrix.
What is the distribution of Q?
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Matrix Normal Distribution
X : (p × n) is a normally distributed matrix with a separablecovariance matrix
X ∼ Np,n (M, Σ, Ψ|m, k) ,
where m = rank(Σ) ≤ p and k = rank(Ψ) ≤ n.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Matrix Normal Distribution
X : (p × n) is a normally distributed matrix with a separablecovariance matrix
X ∼ Np,n (M, Σ, Ψ|m, k) ,
where m = rank(Σ) ≤ p and k = rank(Ψ) ≤ n.
The covariance matrix for X is
cov(X ) = cov(vecX ) = Ψ ⊗ Σ : (pn × pn)
where ⊗ is the Kronecker product.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Matrix Normal Distribution
X : (p × n) is a normally distributed matrix with a separablecovariance matrix
X ∼ Np,n (M, Σ, Ψ|m, k) ,
where m = rank(Σ) ≤ p and k = rank(Ψ) ≤ n.
The covariance matrix for X is
cov(X ) = cov(vecX ) = Ψ ⊗ Σ : (pn × pn)
where ⊗ is the Kronecker product.
Hence, the distribution is singular if m < p and/or k < n.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
We start with the following Theorem.
Theorem
Let Y ∼ Np,n (M, Σ, I |m), where m ≤ p and let A : (n × n) be
a symmetric real matrix of rank r . The characteristic function
of Q = YAY ′ is then
ϕQ(T ) =r
∏
j=1
|I − iλjΓΣ|−1/2etr
1
2iλjΩj(I − iλjΓΣ)−1Γ
,
where T = (tij)pi ,j=1, Γ = (γij) = ((1 + δij) tij)
pi ,j=1
, tij = tji and
δij is the Kronecker delta. The noncentrality parameters are
Ωj = mjm′
j , where mj = M∆j . The vectors ∆j and the values
λj are the latent vectors and roots of A respectively, such that
|∆j | = 1.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Theorem
Suppose Y ∼ Np,n (M, Σ, I |m), where m ≤ p and let
Q = YAY ′, where A : (n × n) is a symmetric real matrix of
rank r . Then the distribution of Q is the same as for
W =r
∑
j=1
λjWj ,
where λj are the nonzero latent roots of A and Wj are
independent noncentral Wishart, i.e.,
Wj ∼ Wp(1, Σ,mjm′
j),
where mj = M∆j and ∆j are the corresponding latent vectors
such that |∆j | = 1 for j = 1, . . . , r . If m < p the noncentral
Wishart distributions are singular.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Definition
If Y ∼ Np,n (M, Σ, I |m), we define the distribution of themultivariate quadratic form Q = YAY ′ to be Qp(A, M, Σ|m).
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Now, suppose that
X ∼ Np,n (M, Σ, Ψ|m, n)
i.e., the columns are dependent as well.
Suppose also that the matrix Σ is non-negative definite of rankm ≤ p. Ψ is positive definite since rank(Ψ) = n.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Theorem
Let X ∼ Np,n (M, Σ, Ψ|m, n), m ≤ p and let A : (n × n) be a
symmetric real matrix of rank r . The distribution of Q = XAX ′
is the same as for
W =r
∑
j=1
λjWj ,
where λj are the nonzero latent roots of Ψ1/2AΨ1/2 and Wj
are independent noncentral Wishart, i.e.,
Wj ∼ Wp(1, Σ,mjm′
j),
where mj = MΨ−1/2∆j and ∆j are the corresponding latent
vectors such that |∆j | = 1 for j = 1, . . . , r . If m < p the
noncentral Wishart distributions are singular.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
We see that
Q = XAX ′ ∼ Qp(Ψ1/2AΨ1/2, MΨ−1/2, Σ|m).
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Theorem
If AΨ is idempotent, then
Qp(Ψ1/2AΨ1/2, MΨ−1/2, Σ) = Wp
(
r , Σ, MAM ′)
,
where r = rank(AΨ).
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Theorem
If AΨ is idempotent, then
Qp(Ψ1/2AΨ1/2, MΨ−1/2, Σ) = Wp
(
r , Σ, MAM ′)
,
where r = rank(AΨ).
Proof.
If AΨ is idempotent, then Ψ1/2AΨ1/2 is idempotent as welland λj = 1 for j = 1, . . . , r and zero otherwise. Use of the factthat the sum of independent Wishart distributed matrices isagain Wishart completes the proof.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Example
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Let us assume that we have a matrix
X =(
X1 X2 · · · Xn
)
∼ Np,n (M, Σ, Ψ) ,
where the expectation is
M = µ1′
and
µ : (p × 1),
1′ = (1, 1, . . . , 1) : (1 × n).
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Assume that Ψ is known and that we want to estimate Σ.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Assume that Ψ is known and that we want to estimate Σ.
For some reason we estimate the mean with µ = 1nX1.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Assume that Ψ is known and that we want to estimate Σ.
For some reason we estimate the mean with µ = 1nX1.
robust for large matrices
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Assume that Ψ is known and that we want to estimate Σ.
For some reason we estimate the mean with µ = 1nX1.
robust for large matrices
it is all we have available, we only know (X − µ1′)
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Matrix Normal Density
The matrix normal density function is
f (X ) = (2π)−12pn|Σ|−n/2|Ψ|−p/2
etr
−1
2Σ−1 (X − M)Ψ−1 (X − M)′
.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
ML Estimators
µml = (1′Ψ−11)−1XΨ−11,
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
ML Estimators
µml = (1′Ψ−11)−1XΨ−11,
nΣ =(
X − µml1′)
Ψ−1(
X − µml1′)
′
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
ML Estimators
µml = (1′Ψ−11)−1XΨ−11,
nΣ =(
X − µml1′)
Ψ−1(
X − µml1′)
′
= X(
Ψ−1 − Ψ−11(
1′Ψ−11)
−11′Ψ−1
)
X ′ = XHX ′
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
ML Estimators
µml = (1′Ψ−11)−1XΨ−11,
nΣ =(
X − µml1′)
Ψ−1(
X − µml1′)
′
= X(
Ψ−1 − Ψ−11(
1′Ψ−11)
−11′Ψ−1
)
X ′ = XHX ′
Theorem
Let X ∼ Np,n (M, Σ, Ψ), where M = µ1′ and Ψ is known.
Then XHX ′ ∼ Wp (n − 1, Σ).
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
But now we have the estimator of Σ as
nΣ =(
X − µ1′)
Ψ−1(
X − µ1′)
′
= XCΨ−1CX ′,
where C is the centralization matrix
C = I − 1(1′1)−11′ = I − n−111′.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
But now we have the estimator of Σ as
nΣ =(
X − µ1′)
Ψ−1(
X − µ1′)
′
= XCΨ−1CX ′,
where C is the centralization matrix
C = I − 1(1′1)−11′ = I − n−111′.
What is the distribution of XCΨ−1CX ′?
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
Theorem
Let X ∼ Np,n
(
µ1′, Σ, Ψ)
, where Ψ is known. The distribution
of XCΨ−1CX ′ is the same as the distribution of
W = W1 + λ∗W ∗,
where W1 and W ∗ are independent and
W1 ∼ Wp (n − 2, Σ) ,
W ∗ ∼ Wp (1, Σ)
and λ∗ = 1 − 1n1′ΨCΨ−11.
Distributions
of Quadratic
Forms
Martin Ohlson
Outline
Introduction
Quadratic
Forms
Univariate
Multivariate
Example
References
References
Graybill, F. (1976). Theory and Application of the Linear Model. DuxburyPress, North Scituate, Massachusetts.
Gupta, A. and Nagar, D. (2000). Matrix Variate Distributions. Chapmanand Hall.
Hayakawa, T. (1966). On the distribution of a quadratic form inmultivariate normal sample. Annals of the Institute of Statistical
Mathematics, 18:191–201.
Khatri, C. (1966). On certain distribution problems based on positivedefinite quadratic functions in normal vectors. The Annals of
Mathematical Statistics, 37(2):468–479.
Mathai, A. and Provost, S. (1992). Quadratic forms in random variables.M. Dekker New York.
Ohlson, M. (2007). Testing Spatial Independence using a Separable
Covariance Matrix. Lic thesis, Linkopings universitet, Linkoping.
Rao, C. (1973). Linear Statistical Inference and Its Applications. JohnWiley & Sons, New York, USA.
Shah, B. (1970). Distribution theory of a positive definite quadratic formwith matrix argument. The Annals of Mathematical Statistics,41(2):692–697.