A normality test for the errors of the linear

model ∗

Miguel A. Arcones

Department of Mathematical Sciences, Binghamton University, Binghamton, NY 13902.

E-mail:arcones@math.binghamton.edu

January 27, 2008

Abstract

We present a normality test for the errors of the linear regression model based on thenormality test in Epps and Pulley (1983). We prove that the presented test is consistentagainst any alternative. We show that the test statistic under the null hypothesis isasympotically equivalent to a degenerate V–statistic. We also consider the asymptoticdistribution of the test under contiguous alternatives.

1 Introduction

Many of the statistical methods in linear regression assume that the residuals have a normal

distribution. In this paper, we study a normality test for the residuals of a linear regression

model. Many authors have studied normality tests for (independent identically distributed)

i.i.d. observations. Reviews of normality tests are Henze (2002) and Mecklin and Mundfrom

(2004). Classical normality tests are the ones by Shapiro and Wilk (1965, 1968) and Epps

and Pulley (1983). Several authors have considered tests of normality for the linear regression

model. Jureckova, Picek, and Sen (2003) and Sen, Jureckova, and Picek (2003) considered a

test of normality for the residuals of a linear regression model using the approach in Shapiro

and Wilk. Here, we apply the normality test in Epps and Pulley (1983) to the linear regression

model. The asymptotic distribution of the test statistic of the Epps and Pulley test under

(independent identically distributed random variables) i.i.d.r.v.’s was obtained by Baringhaus

and Henze (1988). The test in Epps and Pulley (1983) is known in the literature as the BHEP

test.

∗Key words and phrases: normality test, linear regression.

1

Suppose that X1, . . . , Xn are i.i.d.r.v.’s from a (cumulative distribution function) c.d.f. F .

Assume that X1 has a second finite moment, then we may define µ = E[X1] and σ2 = E[(X1−µ)2]. We have that X1 has a nondegenerate normal distribution if and only if σ−1(X1−µ) has

a standard normal distribution. This is equivalent to E[exp(itσ−1(X1 − µ))] = exp(−2−1t2),

for each t ∈ R. The BHEP test is significative for nonnormality for large values of the statistic

Tn := n

∫

R|n−1

n∑j=1

exp(itσ−1n (Xj − Xn))− exp(−2−1t2)|2φδ(t) dt, (1.1)

where Xn = n−1∑n

j=1 Xj, σ2n = n−1

∑nj=1(Xj − Xn)2 and φδ(t) = (2πδ2)−1/2 exp(−2−1δ−2t2).

Observe that in (1.1) the modulus of the complex number n−1∑n

j=1 exp(itσ−1n (Xj − Xn)) −

exp(−2−1t2) is taken. From now, we will call the normality test based on the statistic in

(1.1), the BHEP(δ) test. φδ is the (probability density function) p.d.f. of a normal r.v. with

mean zero and variance δ2 > 0. By tuning the parameter δ is possible to increase the power

of the test (see Henze and Zirkler, 1990). Henze and Zirkler (1990) proved that for some

distributions the BHEP test is more powerful when δ = 1/2.

We will use the usual multivariate notation. For example, given u = (u1, . . . , ud)′ ∈ Rd

and v = (v1, . . . , vd)′ ∈ Rd, u′v =

∑dj=1 ujvj and |u| = (

∑nj=1 u2

j)1/2. Given a d × d matrix

A, ‖A‖ = supv1,v2∈Rd,|v1|,|v2|=1 v′1Av2 and trace(A) =∑d

j=1 aj,j, where A = (aj,k)1≤j,k≤d. Given

θ ∈ Rd and ε > 0, B(θ, ε) = {x ∈ Rd : |x− θ| < ε}. Id will denote the identity matrix in Rd.

c will denote an universal constant which may change from occurrence to occurrence.

In this paper, we present a test of normality for the errors of a linear regression model.

We consider the linear regression model: Yj = x′n,jβ + εj, j ≥ 1, where {εj}∞j=1 is a sequence

of i.i.d.r.v.’s with mean zero; xn,j, 1 ≤ j ≤ n are p dimensional vectors and β ∈ Rp is a

parameter to be estimated. xn,j is called the regressor or predictor variable. Yj is called the

response variable. εj is an error variable. Let F be the c.d.f. of the sequence {εj}. We assume

that F has mean zero. We study the testing problem

H0 : F has a normal distribution, versus H1 : F does not. (1.2)

Assuming the condition:

(A) An :=∑n

j=1 xn,jx′n,j is a nonsingular p× p matrix,

the least squares estimator of β is βn = A−1n (

∑nj=1 Yjxn,j). To estimate the distribution F

of the errors we use the residuals εj = Yj − β′nxn,j, 1 ≤ j ≤ n. Let

Dn,δ :=

∫

R|n−1

n∑j=1

exp(itσ−1n εj)− exp(−2−1t2)|2φδ(t) dt (1.3)

= ‖n−1

n∑j=1

exp(itσ−1n εj)− exp(−2−1t2)‖2

L2(φδ),

2

where σ2n := (n − p)−1

∑nj=1 ε2

j . As it is well known σ2n is an unbiased estimator of σ2

F :=

VarF (ε). We abbreviate Dn,δ to Dn. Note that Dn estimates

DF :=

∫

R|EF [exp(itσ−1

F ε)]− exp(−2−1t2)|2φδ(t) dt.

Given 1 > α > 0, let

an,δ,α = inf{λ ≥ 0 : PΦ{Dn ≤ λ} ≥ α},where PΦ is the probability measure when the errors have a standard normal distribution. We

abbreviate an,δ,α to an,α. Since

βn = A−1n

n∑j=1

Yjxn,j = A−1n

n∑j=1

xn,j(x′n,jβ + εj) = β + A−1

n

n∑j=1

εjxn,j (1.4)

and

εj = Yj− β′nxn,j = εj−(βn−β)′xn,j = εj−(n∑

k=1

εkxn,k)′A−1

n xn,j = εj−n∑

k=1

x′n,kA−1n xn,jεk, (1.5)

the distribution of Dn does not depend on σ. Observe that since the distribution of the

residuals depends on the design of regressors, the distribution of the test statistic depends on

the design of regressors. Given a design of regressors xn,1, . . . , xn,n, the value of an,α can be

estimated via simulations. Therefore,

the test rejects the null hypothesis if Dn > an,α (1.6)

can be implemented.

In Section 2, we present the asymptotics of the test statistic Dn. We show that under

certain conditions, the test is consistent against any alternative hypothesis. We obtain the

limit distribution of nDn under the null hypothesis and under contiguous alternatives. We

show that nDn under the null hypothesis is asympotically equivalent to a degenerate V–

statistic. In particular, we get that nan,α converges to a limit. This implies that for n large

enough, the test can be done using limn→∞ nan,α. In Section 3, we present the outcome of

some simulations comparing the presented test with the test in Shapiro and Wilk (1965).

The result of the simulations show that the presented test is competitive. The proofs of the

theorems in Section 2 are in Section 4.

2 Asymptotics of the test statistic

First, we give a useful representation of test statistic.

3

Lemma 2.1. We have that

Dn (2.1)

= n−2

n∑

j,k=1

exp(−2−1δ2σ−2n (εj − εk)

2)

−2(1 + δ2)−1/2n−1

n∑j=1

exp(−2−1δ2(δ2 + 1)−1σ−2n ε2

j) + (1 + 2δ2)−1/2.

From the previous representation it is easy to compute the test statistic.

The following theorem gives the consistency of the test.

Theorem 2.1. Consider the linear regression model with i.i.d. random errors {εj}. Suppose

that EF [ε1] = 0 and EF [ε41] < ∞, where F is the c.d.f. of ε1. Assume that the regressors xn,j,

1 ≤ j ≤ n, satisfy condition (A). Then,

DnP→ DF :=

∫ ∞

−∞|EF [eitσ−1

F ε]− e−2−1t2|2φδ(t) dt,

where σ2F = VarF (ε).

We have that DF = 0 if and only if F is a normal distribution. Hence, for each 0 < α < 1,

an,α → 0, as n →∞. If F is not a normal c.d.f., then PF{Dn > an,α} → 1, as n →∞.

By Lemma 2.1, Dn is a V–statistic with an estimated parameter. We will see that nDn

is a asymptotically equivalent to a regular V–statistic. So, in order to present the limit

distribution of nDn under the null hypothesis, we need to present some results about U–

statistics. General references on U–statistics are Lee (1990) and de la Pena and Gine (1999).

Given a sequence of i.i.d.r.v.’s {Xj} with values in a measurable space (S,S) and a measurable

function h : S × S → R, the U-statistic (of order 2) with kernel h is

Un(h) :=(n− 2)!

n!

∑

1≤j 6=k≤n

h(Xj, Xk).

The V–statistic with kernel h is

Vn(h) := n−2

n∑

j,k=1

h(Xj, Xk).

A function h : S×S → R is called symmetric if for each x, y ∈ S, h(x, y) = h(y, x). A function

h : S × S → R is P–degenerate if E[h(x,X1)] = E[h(X1, x)] = E[h(X1, X2)] a.s. [P ], where

P := L(X1) is the law of X1. By Theorem 3.2.2.1 in Lee (1990, page 79), if h is a symmetric

degenerate kernel with E[h(X1, X2)] = 0 and E[(h(X1, X2))2] < ∞, then

n−1∑

1≤j 6=k≤n

h(Xj, Xk)d→

∞∑

k=1

λk(g2k − 1),

4

where {gk} is a sequence of i.i.d.r.v.’s with a standard normal distribution and {λk} denotes

the eigenvalues of the integral operator Qh : L2(S,S, P ) → L2(S,S, P ) defined by Qhq(x) =

E[h(x,X1)q(X1)], where L2(S,S, P ) is the collection of measurable functions q : (S,S) → Rsuch that E[(q(X1))

2] < ∞. It follows from the previous result and the law of the large

numbers that if h is a symmetric degenerate kernel with E[h(X1, X2)] = 0, E[(h(X1, X2))2] <

∞ and E[|h(X1, X1)|] < ∞, then

n−1

n∑

j,k=1

h(Xj, Xk)d→

∞∑

k=1

λk(g2k − 1) + E[h(X1, X1)], (2.2)

where {λk} and {gk} are as before.

In order to get the asymptotic distribution of the test, we assume the condition:

(B) n−1

n∑

j,k=1

x′n,jA−1n xn,k → 1 as n →∞.

Theorem 2.2. Consider the linear regression model with i.i.d. random errors {εj}, which

have a normal distribution with mean zero and variance σ2 > 0. Assume that the regressors

xn,j, 1 ≤ j ≤ n, satisfy conditions (A) and (B).

Then,

nDn

−‖n−1/2∑n

j=1(cos(σ−1tεj) + sin(σ−1tεj)− e−2−1t2(1 + σ−1tεj − 2−1σ−2t2(ε2

j − σ2))) ‖2

L2(φδ)Pr→ 0.

Besides,

‖n−1/2∑n

j=1(cos(σ−1tεj) + sin(σ−1tεj)− e−2−1t2(1 + σ−1tεj − 2−1σ−2t2(ε2

j − σ2))) ‖2

L2(φδ)

=∑n

j,k=1 h(σ−1εj, σ−1εk)

d→ ∑∞k=1 λk(g

2k − 1) + EΦ[h(ε, ε)],

where {gk} is a sequence of i.i.d.r.v.’s with a standard normal distribution and {λk} de-

notes the eigenvalues of the operator Qh : L2(S,S, Φ) → L2(S,S, Φ) defined by Qhq(x) =

EΦ[h(x, ε)q(ε)], where Φ is the law of a standard normal r.v. and

h(a, b) (2.3)

= exp(−2−1δ2(a− b)2)

−(δ2 + 1)−1/2 exp(−2−1(δ2 + 1)−1δ2a2)

× (1 + (δ2 + 1)−1δ2b− 2−1(δ2 + 1)−2δ2(b2 − 1) (1− (δ2 + 1)−2δ2a2))

−(δ2 + 1)−1/2 exp(−2−1(δ2 + 1)−1δ2b2)

× (1 + (δ2 + 1)−1δ2a− 2−1(δ2 + 1)−2δ2(a2 − 1) (1− (δ2 + 1)−2δ2b2))

+(2δ2 + 1)−1/2 (1 + (2δ2 + 1)−1δ2(1− 2−1(a− b)2) + 2−23(2δ2 + 1)−2δ4(a2 − 1)(b2 − 1)) ,

5

It is of interest to know the distribution of nDn in the case of sequence of contiguous

alternatives. Given a sequence of measurable space spaces {(Sn,Sn)} and two sequences

of probabilities measures {Pn} and {Qn} such that Pn and Qn are probability measures on

(Sn,Sn), it is said that {Qn} is contiguous to {Pn} if Pn(An) → 0 implies Qn(An) → 0 for each

sequence of sets {An}, An ∈ Sn (see e.g. Section VI.1 in Hajek and Sidak, 1967). Suppose that

the observations (xn,j, Yn,j), 1 ≤ j ≤ n, satisfy Yn,j = β′xn,j + εn,j, j ≥ 1, where εn,1, . . . , εn,n

are i.i.d.r.v.’s with pdf

fn(x) = φσ(x)(1 + n−1/2αn(x)), (2.4)

where σ > 0. As before, define βn = A−1n (

∑nj=1 Yn,jxn,j), εn,j = Yn,j − β′nxn,j, 1 ≤ j ≤ n,

σ2n := (n − p)−1

∑nj=1 ε2

n,j and Dn,δ is as in (1.3). Assume that the following condition is

satisfied

(C) There exists a function α : R→ R such that∫

R(αn(x)− α(x))2φσ(x) dx → 0

and ∫

R(α(x))2φσ(x) dx < ∞.

Let {εj} be sequence of i.i.d.r.v.’s with a normal distribution with mean zero and variance

σ2. Let Pn = L(ε1, . . . , εn) be the law of (ε1, . . . , εn) in (Rn,B(Rn)), where B(Rn) is the Borel

σ–field of Rn. Let Qn = L(εn,1, . . . , εn,n) be the law of (εn,1, . . . , εn,n) in (Rn,B(Rn)). We

claim that under Condition (C), the sequence of probability measures {Qn} is contiguous to

the sequence {Pn}. Let pn be the p.d.f. of Pn with respect to the Lebesgue measure in Rn.

Let qn be the p.d.f. of Qn with respect to the Lebesgue measure in Rn. Under Condition (C),

n−1/2∑n

j=1 αn(εj)d→ N(0, τ 2), where τ 2 =

∫R α2(x)φσ(x) dx. By problem VI.5 in page 238 of

Hajek and Sidak (1967) this implies that

logqn(ε1, . . . , εn)

pn(ε1, . . . , εn)

d→ N(2−1τ 2, τ 2).

Hence, by the LeCam’s first lemma (see e.g. Section VI.1.2 in Hajek and Sidak, 1967) {Qn}is contiguous to {Pn}.

The convergence of U–statistics was considered by Gregory (1977). Let X1, . . . , Xn be

i.i.d.r.v.’s with law P and let h be a symmetric P–degenerate kernel with E[(h(X1, X2))2] < ∞.

Suppose that Xn,1, . . . , Xn,n are i.i.d.r.v.’s with law Pn, where Pn is absolutely continuous with

respect to P with Radon–Nikodym derivative dPn

dP= 1 + n−1/2αn, where αn ∈ L2(S,S, P ) and

αn → α in L2(S,S, P ). By Theorem 2.1 in Gregory (1977) if h is a symmetric degenerate

kernel with E[h(X1, X2)] = 0 and E[(h(X1, X2))2] < ∞, then

n−1∑

1≤j 6=k≤n

h(Xn,j, Xn,k)d→

∞∑j=1

λj((gj + aj)2 − 1),

6

where {gj} is a sequence of i.i.d.r.v.’s with a standard normal distribution, aj = E[ψj(X1)α(X1)],

and {(λj, ψj)} is the finite or infinite collection of pairs of eigenvalues–orthonormal eigenvec-

tors of the operator Qh. Assuming that E[|h(X1, X1)|] < ∞, by the strong of the large

numbers,

n−1

n∑j=1

(h(Xj, Xj)− E[h(Xj, Xj)]) → 0 a.s.

Hence, by contiguity

n−1

n∑j=1

(h(Xn,j, Xn,j)− E[h(Xj, Xj)])Pr→ 0.

Hence, if h is a symmetric degenerate kernel with E[h(X1, X2)] = 0, E[(h(X1, X2))2] < ∞

and E[|h(X1, X1)|] < ∞, then

n−1

n∑

j,k=1

h(Xn,j, Xn,k)d→

∞∑j=1

λj((gj + aj)2 − 1) + E[h(X1, X1)]. (2.5)

Theorem 2.3. Under the notation above, suppose that:

(i) The regressors xn,j, 1 ≤ j ≤ n, satisfy conditions (A) and (B).

(ii) The errors εn,1, . . . , εn,n are i.i.d.r.v.s with p.d.f. given by (2.4).

(iii) Condition (C) is satisfied.

Then,

nDn (2.6)

−‖n−1/2∑n

j=1

(cos(σ−1tεn,j) + sin(σ−1tεn,j)− e−2−1t2

(1 + σ−1tεj − 2−1σ−2t2(ε2

j − σ2))) ‖2

L2(φδ)

Pr→ 0.

Besides,

‖n−1/2∑n

j=1(cos(σ−1tεn,j) + sin(σ−1tεn,j)− e−2−1t2(1 + σ−1tεj − 2−1σ−2t2(ε2

j − σ2))) ‖2

L2(φδ)

= n−1∑n

j,k=1 h(σ−1εn,j, σ−1εn,k)

d→ ∑∞j=1 λj((gj + aj)

2 − 1) + EΦ[h(ε, ε)],

where h is as in (2.3), {gj} is a sequence of i.i.d.r.v.’s with a standard normal distribution,

aj = EΦ[ψj(ε)α(ε)], and {(λj, ψj)} is the finite or infinite collection of pairs of eigenvalues-

orthonormal eigenvectors of the operator Qh.

3 Simulations.

We have made simulations to estimate the power of the (Shapiro–Wilk) SW test and the

presented test for several distributions. We have used xn,j = (1, j)′, for 1 ≤ j ≤ n, and σ = 1.

7

Besides the test introduced in the introduction, we also consider the test using the biased

estimator of the variance of the error. Let

Dn,δ,bsd := ‖n−1

n∑j=1

exp(itσ−1n,bsdεj)− exp(−2−1t2)‖2

L2(φδ), (3.1)

where σ2n,bsd := n−1

∑nj=1 ε2

j . The test rejects H0, for Dn,bsd,δ ≥ bn,δ,α, where

bn,δ,α = inf{λ ≥ 0 : PΦ{Dn,δ,bsd ≤ λ} ≥ α}.

It is easy to see that theorems 2.1-2.3 hold for Dn,δ,bsd.

The following tables show the values of nan,,δα and nbn,δ,α for some values of n. The tables

were obtained by doing 10000 simulations from a standard normal distribution. The test

statistic Dn was found using the expression given by Lemma 2.1.

nan,1,α α = 0.10 α = 0.05 α = 0.01

n = 6 0.1723317 0.2263769 0.3625338

n = 8 0.2065039 0.2837184 0.4601209

n = 10 0.2220333 0.2971192 0.4734248

n = 15 0.2555162 0.3432884 0.5263078

n = 20 0.2599958 0.3440845 0.5340384

n = 25 0.2714118 0.3549643 0.5689468

nan,1/2,α α = 0.10 α = 0.05 α = 0.01

n = 6 0.01901845 0.02468509 0.03896278

n = 8 0.02047842 0.02686428 0.04253534

n = 10 0.02256076 0.03003705 0.04934944

n = 15 0.02546400 0.03517977 0.06237715

n = 20 0.02679697 0.03730412 0.06435028

n = 25 0.02841144 0.03974311 0.06939096

nbn,1,α α = 0.10 α = 0.05 α = 0.01

n = 6 0.2512835 0.3084409 0.4880363

n = 8 0.2659785 0.3372054 0.4981566

n = 10 0.2709936 0.3503190 0.5136096

n = 15 0.2758713 0.3571201 0.5456935

n = 20 0.2849200 0.3653052 0.5585711

n = 25 0.2895949 0.3711702 0.5712650

8

nbn,1/2,α α = 0.10 α = 0.05 α = 0.01

n = 6 0.01831819 0.02512786 0.04152145

n = 8 0.02142009 0.03030869 0.05221142

n = 10 0.02346332 0.03272659 0.05518262

n = 15 0.02610572 0.03617368 0.06352592

n = 20 0.02789834 0.03964410 0.06633810

n = 25 0.02827096 0.04006235 0.07027089

Since nDn,δ,unbsd and nDn,δ,bsd converge in distribution to the same limit, we have that

nan,α,δ and nbn,α,δ are close for n large.

The following table shows the power when α = 0.05 of the tests of normality in Shapiro

and Wilk (1965) and in Epps and Pulley (1983) (using δ = 1 and δ = 1/2 and the unbiased

and the biased estimators of the variance).

SW BHEPunbsd(1) BHEPunbsd(1/2) BHEPbsd(1) BHEPbsd(1/2)

n = 6 0.1098 0.0989 0.0929 0.1039 0.1050

n = 8 0.1807 0.1756 0.1782 0.1915 0.1859

n = 10 0.2719 0.2761 0.2621 0.2777 0.2877

n = 15 0.5124 0.4768 0.4715 0.5105 0.5162

n = 20 0.7022 0.6753 0.6490 0.6823 0.6661

n = 25 0.8370 0.8056 0.7721 0.8018 0.8070

Alternative: exponential distribution with mean one

SW BHEPunbsd(1) BHEPunbsd(1/2) BHEPbsd(1) BHEPbsd(1/2)

n = 6 0.0755 0.0789 0.0652 0.0737 0.0740

n = 8 0.0915 0.1013 0.1117 0.0977 0.1059

n = 10 0.1057 0.1433 0.1423 0.1169 0.1404

n = 15 0.1719 0.1910 0.2148 0.1752 0.2015

n = 20 0.2280 0.2625 0.2626 0.2263 0.2294

n = 25 0.2748 0.3119 0.2931 0.2719 0.2829

Alternative: double exponential distribution

SW BHEPunbsd(1) BHEPunbsd(1/2) BHEPbsd(1) BHEPbsd(1/2)

n = 6 0.1471 0.1739 0.1459 0.1523 0.1585

n = 8 0.3058 0.3288 0.3462 0.3199 0.3344

n = 10 0.4106 0.4898 0.4990 0.4436 0.4872

n = 15 0.6922 0.7143 0.7035 0.7045 0.6892

n = 20 0.8181 0.8425 0.8353 0.8272 0.8008

n = 25 0.9006 0.9139 0.8836 0.8978 0.8765

Alternative: Cauchy distribution

9

SW BHEPunbsd(1) BHEPunbsd(1/2) BHEPbsd(1) BHEPbsd(1/2)

n = 6 0.0596 0.0583 0.0505 0.0589 0.0586

n = 8 0.0646 0.0649 0.0706 0.0681 0.0703

n = 10 0.0629 0.0828 0.0836 0.0677 0.0830

n = 15 0.0882 0.0900 0.1115 0.0877 0.1060

n = 20 0.1039 0.1180 0.1281 0.1026 0.1172

n = 25 0.1242 0.1300 0.1491 0.1114 0.1414

Alternative: logistic(1) distribution

SW BHEPunbsd(1) BHEPunbsd(1/2) BHEPbsd(1) BHEPbsd(1/2)

n = 6 0.0636 0.0506 0.04547 0.0579 0.0541

n = 8 0.0703 0.0605 0.0578 0.0689 0.0587

n = 10 0.0844 0.0774 0.0632 0.0878 0.0715

n = 15 0.1413 0.0970 0.0825 0.1462 0.1069

n = 20 0.2247 0.1646 0.1141 0.2051 0.1415

n = 25 0.3171 0.2304 0.1360 0.2837 0.1871

Alternative: Beta(2, 1) distribution

SW BHEPunbsd(1) BHEPunbsd(1/2) BHEPbsd(1) BHEPbsd(1/2)

n = 6 0.0565 0.0421 0.0339 0.0491 0.0444

n = 8 0.0570 0.0285 0.0305 0.0417 0.0343

n = 10 0.0556 0.0322 0.0232 0.0468 0.0283

n = 15 0.0840 0.0280 0.0133 0.0670 0.0223

n = 20 0.1318 0.0477 0.0106 0.0948 0.0162

n = 25 0.2182 0.0603 0.0105 0.1374 0.0200

Alternative: uniform (0, 1) distribution

The previous table seems to indicate that the presented tests are competitive with the

known tests. The BHEPunbsd(1) test is the most powerful for the double exponential, Cauchy

and logistic distributions.

4 Proofs

We will use the following lemma:

Lemma 4.1. Under condition (A),

(i) For each 1 ≤ j ≤ n, x′n,jA−1n xn,j ≥ 0.

(ii)∑n

j=1 x′n,jA−1n xn,j = p.

Proof. Under condition (A), An is a symmetric positive definite matrix. So, A−1/2n exists and

x′n,jA−1n xn,j = |A−1/2

n xn,j|2 ≥ 0, i.e. (i) holds.

10

Consider the linear regression model with i.i.d. errors with mean zero and variance one.

It is well known (see e.g. Section 7.2 in Neter et al., 1996) that the linear regression model

can be written as

Y = Xβ + ε,

where Y = (Y1, . . . , Yn)′, ε = (ε1, . . . , εn)′, β = (β1, . . . , βp)′,

X =

xn,1,1 xn,1,2 · · · xn,1,p

xn,2,1 xn,2,2 · · · xn,2,p

· · · · · ·· · · · · ·· · · · · ·

xn,n,1 xn,n,2 · · · xn,n,p

.

and xn,j = (xn,j,1, . . . , xn,j,p)′. With this matrix notation,

An = X′X,

β = (X′X)−1X′Y

and

ε = Y −Xβ = ε−X(X′X)−1X′ε = (In −X(X′X)−1X′)ε.

Assuming that for each 1 ≤ j ≤ n, E[εj] = 0 and E[ε2j ] = 0, we have that

E[εε′] = (In −X(X′X)−1X′)(In −X(X′X)−1X′) = In −X(X′X)−1X′.

So,

∑nj=1 Var(εj) = trace(In −X(X′X)−1X′) (4.1)

= n− trace((X′X)−1X′X) = n− trace(Ip) = n− p.

By (1.5)

Var(εj) = (1− x′n,jA−1n xn,j)

2 +∑

k:1≤k≤n,k 6=j(x′n,kA

−1n xn,j)

2

= 1− 2x′n,jA−1n xn,j +

∑nk=1(x

′n,kA

−1n xn,j)

2

= 1− 2x′n,jA−1n xn,j +

∑nk=1 x′n,jA

−1n xn,kx

′n,kA

−1n xn,j = 1− x′n,jA

−1n xn,j

andn∑

j=1

Var(εj) = n−n∑

j=1

x′n,jA−1n xn,j. (4.2)

Hence, (ii) follows from (4.1) and (4.2).

11

Proof of Lemma 2.1. Using that

∫

Rexp(it)φδ(t) dt = exp(−2−1δ2),

we have that

Dn

= n−2∑n

j,k=1

∫R(exp(itσ−1

n (εj − εk))− 2 exp(itσ−1n εj)e

−2−1t2 + e−t2)φδ(t) dt

= n−2∑n

j,k=1

∫R exp(itσ−1

n (εj − εk))φδ(t) dt

−2(1 + δ2)−1/2n−1∑n

j=1

∫R exp(itσ−1

n εj)φδ(1+δ2)−1/2(t) dt

+∫R(2δ

2 + 1)−1/2φδ(2δ2+1)−1/2(t) dt

= n−2∑n

j,k=1 exp(−2−1δ2σ−2n (εj − εk)

2)

−2(1 + δ2)−1/2n−1∑n

j=1 exp(−2−1δ2(δ2 + 1)−1σ−2n ε2

j) + (1 + 2δ2)−1/2.

¤Proof of Theorem 2.1. We have that

DF =∫∞−∞ |EF [eitσ−1

F ε]− e−2−1t2|2φδ(t) dt

=∫∞−∞ EF [exp(itσ−1

F (ε1 − ε2))− 2 exp(itσ−1F ε1 − 2−1t2) + exp(2−1t2)]φδ(t) dt

= EF [exp(−2−1δ2σ−2F (ε1 − ε2)

2)]

−2(1 + δ2)−1/2EF [exp(−2−1δ2(δ2 + 1)−1σ−2F ε2

1)] + (1 + 2δ2)−1/2.

Using (2.1) and the previous equality, it suffices to prove that

n−2

n∑

j,k=1

exp(−2−1δ2σ−2n (εj − εk)

2)Pr→ EF [exp(−2−1δ2σ−2

F (ε1 − ε2)2)] (4.3)

and

n−1

n∑j=1

exp(−2−1δ2(δ2 + 1)−1σ−2n ε2

j)Pr→ EF [exp(−2−1δ2(δ2 + 1)−1σ−2

F ε21)]. (4.4)

First, we prove that

n−1

n∑

j,k=1

x′n,jA−1n xn,kεjεk

Pr→ 0. (4.5)

Equation (4.5) follows from the following (using Lemma 4.1):

EF [n−1

n∑j=1

x′n,jA−1n xn,jε

2j ] = σ2

F n−1

n∑j=1

x′n,jA−1n xn,j = σ2

F n−1p → 0,

EF [n−1∑

1≤j<k≤n

x′n,jA−1n xn,kεjεk] = 0,

12

and

VarF (n−1∑

1≤j<k≤n x′n,jA−1n xn,kεjεk) = (VarF (ε2

1))2n−2

∑1≤j<k≤n(x′n,jA

−1n xn,k)

2

≤ 2−1(VarF (ε21))

2n−2∑n

j,k=1 x′n,jA−1n xn,kx

′n,kA

−1n xn,j

= 2−1(VarF (ε21))

2n−2∑n

j=1 x′n,jA−1n xn,j = 2−1(VarF (ε2

1))2n−2p → 0.

Next, we prove that σ2n

Pr→ σ2F . By (1.4),

n−1

n∑j=1

(βn − β)′xn,jεj = n−1

n∑j=1

(n∑

k=1

xn,kεk)′A−1

n xn,jεj = n−1

n∑

j,k=1

x′n,jA−1n xn,kεjεk

and

n−1∑n

j=1((βn − β)′xn,j)2 = n−1

∑nj=1(βn − β)′xn,jx

′n,j(βn − β) (4.6)

= n−1(βn − β)′An(βn − β) = n−1(∑n

j=1 xn,jεj)A−1n (

∑nk=1 xn,kεk)

= n−1∑n

j,k=1 x′n,jA−1n xn,kεjεk.

Using the previous expressions,

n−1(n− p)σ2n (4.7)

= n−1∑n

j=1(εj − (βn − β)′xn,j)2

= n−1∑n

j=1 ε2j − 2n−1

∑nj=1(βn − β)′xn,jεj + n−1

∑nj=1((βn − β)′xn,j)

2

= n−1∑n

j=1 ε2j − n−1

∑nj,k=1 x′n,jA

−1n xn,kεjεk.

Hence, by (4.5) and (4.7),

σ2n

Pr→ σ2F . (4.8)

Next, we prove that

n−2

n∑

j,k=1

|(εj − εk)2 − (εj − εk)

2| Pr→ 0. (4.9)

Using (1.5), we have that

n−2∑n

j,k=1 |(εj − εk)2 − (εj − εk)

2|= n−2

∑nj,k=1 | − 2(εj − εk)(βn − β)′(xn,j − xn,k) + ((βn − β)′(xn,j − xn,k))

2|≤ 2(n−2

∑nj,k=1(εj − εk)

2)1/2(n−2∑n

j,k=1((βn − β)′(xn,j − xn,k))2)1/2

+n−2∑n

j,k=1((βn − β)′(xn,j − xn,k))2.

By the law of the large numbers for U–statistics,

n−2

n∑

j,k=1

(εj − εk)2 = n−2

∑

1≤j 6=k≤n

(εj − εk)2 Pr→ E[(ε1 − ε2)

2]. (4.10)

13

By (4.6) and (4.5),

n−2∑n

j,k=1((βn − β)′(xn,j − xn,k))2 ≤ 4n−2

∑nj,k=1((βn − β)′xn,j)

2

= 4n−1∑n

j,k=1 x′n,jA−1n xn,kεjεk

Pr→ 0.

Hence, (4.9) follows.

Using that for a, b ≥ 0, |e−a − e−b| ≤ |a− b| and (4.8)–(4.10),

|n−2∑n

j,k=1 exp(−2−1δ2σ−2n (εj − εk)

2)− n−2∑n

j,k=1 exp(−2−1δ2σ−2F (εj − εk)

2)| (4.11)

≤ 2−1δ2|σ−2n − σ−2

F |n−2∑n

j,k=1(εj − εk)2

≤ 2−1δ2|σ−2n − σ−2

F |n−2∑n

j,k=1 |(εj − εk)2 − (εj − εk)

2|+2−1δ2|σ−2

n − σ−2F |n−2

∑nj,k=1(εj − εk)

2 Pr→ 0.

By a similar argument,

|n−2∑n

j,k=1 exp(−2−1δ2σ−2F (εj − εk)

2)− n−2∑n

j,k=1 exp(−2−1δ2σ−2F (εj − εk)

2)| (4.12)

≤ 2−1δ2σ−2F |n−2

∑nj,k=1 |(εj − εk)

2 − (εj − εk)2| Pr→ 0.

By the law of the large numbers for V–statistics,

n−2∑n

j,k=1 exp(−2−1δ2σ−2F (εj − εk)

2)Pr→ EF [exp(−2−1δ2σ−2

F (ε1 − ε2)2)]. (4.13)

Hence, (4.3) follows from (4.11)–(4.13). The proof of (4.4) is similar and it is omitted. ¤

Theorem 2.3 with αn ≡ 0 implies Theorem 2.2. Hence, we only prove Theorem 2.3.

Lemma 4.2. Under the conditions in Theorem 2.3,

A1/2n (βn − β) = OP (1), (4.14)

n−1/2

n∑j=1

(ε2n,j − σ2) = OP (1) (4.15)

and

n1/2(σ2n − σ2)− n−1/2

n∑j=1

(ε2n,j − σ2)

Pr→ 0. (4.16)

Proof. It is easy to see that

n1/2E[εn,1] →∫

Rxα(x)φσ(x) dx, (4.17)

n1/2(E[ε2

n,1]− σ2) →

∫

Rx2α(x)φσ(x) dx. (4.18)

14

and

n1/2(E[ε4

n,1]− 3σ4) →

∫

Rx4α(x)φσ(x) dx. (4.19)

Using (1.4), we get that

|A1/2n (βn − β)|2 = (βn − β)′An(βn − β) =

n∑

j,k=1

x′n,jA−1n xn,kεn,jεn,k. (4.20)

Using Lemma 4.1 and (4.17)–(4.19), we get that

E[n∑

j=1

x′n,jA−1n xn,jε

2n,j] = E[ε2

n,1]n∑

j=1

x′n,jA−1n xn,j = pE[ε2

n,1] = 0(1),

E[∑

1≤j<k≤n x′n,jA−1n xn,kεn,jεn,k] ≤ 2−1(E[εn,1])

2∑n

j,k=1 x′n,jA−1n xn,k

≤ 2−2(E[εn,1])2∑n

j,k=1(x′n,jA

−1n xn,j + x′n,kA

−1n xn,k)

= 2−1pn(E[εn,1])2 = 0(1),

and

Var(∑

1≤j<k≤n x′n,jA−1n xn,kεn,jεn,k) = n−1

∑1≤j<k≤n Var(x′n,jA

−1n xn,kεn,jεn,k)

=∑

1≤j<k≤n(x′n,jA−1n xn,k)

2Var(εn,j)Var(εn,k)

≤ 2−1∑n

j,k=1(x′n,jA

−1n xn,k)

2(Var(ε2n,1))

2

= 2−1(Var(ε2n,1))

2∑n

j,k=1 x′n,jA−1n xn,kx

′n,kA

−1n xn,j

= 2−1(Var(ε2n,1))

2∑n

j=1 x′n,jA−1n xn,j = 2−1(Var(ε2

n,1))2p = 0(1).

Hence, (4.14) follows.

By (4.18) and (4.19),

E[n−1/2

n∑j=1

(ε2n,j − σ2)] = O(1)

and

Var(n−1/2

n∑j=1

(ε2n,j − σ2)) = OP (1).

So, (4.15) holds.

By (4.7),

σ2n = (n− p)−1

n∑j=1

ε2n,j − (n− p)−1

n∑

j,k=1

x′n,jA−1n xn,kεn,jεn,k.

Hence, to prove (4.16), it suffices to prove that

n−3/2

n∑j=1

ε2n,j

Pr→ 0 (4.21)

15

and

n−1/2

n∑

j,k=1

x′n,jA−1n xn,kεn,jεn,k

Pr→ 0. (4.22)

By (4.18),

E[n−3/2

n∑j=1

ε2n,j] → 0

which implies (4.21). (4.22) follows from (4.14) and (4.20).

Lemma 4.3. Under the conditions in Theorem 2.3, for each 0 < M < ∞,

sup|h|≤M,|v|≤M

‖Un(t, h, v)− Un(t, 0, 0)‖L2(φδ)Pr→ 0, (4.23)

where

Un(t, h, v)

:= n−1/2∑n

j=1(g(εn,j, t, h′A−1/2

n xn,j, (σ2 + n−1/2v)1/2)−Gn(t, h′A−1/2

n xn,j, (σ2 + n−1/2v)1/2)),

is defined for n > M2σ−4, t ∈ R, |h| ≤ M , |v| ≤ M ;

g(x, t, µ, s) := cos(s−1t(x− µ)) + sin(s−1t(x− µ))− e−2−1t2 , t, µ ∈ R, s > 0,

and Gn(t, µ, s) := E[g(εn,1, t, µ, s)], t, µ ∈ R, s > 0.

Proof. We say that a function f : A → B has finite range if the cardinality of f(A) is finite.

Given η > 0, take π1 : {h ∈ Rp : |h| ≤ M} → {h ∈ Rp : |h| ≤ M} with finite range such that

|π1(h) − h| ≤ η for each |h| ≤ M ; and take π2 : [−M, M ] → [−M, M ] with finite range such

that |π2(v)− v| ≤ η for each |v| ≤ M . Given τ > 0,

Pr{sup|h|,|v|≤M ‖Un(t, h, v)− Un(t, 0, 0)‖L2(φδ) ≥ τ} (4.24)

≤ Pr{sup|h|,|v|≤M ‖Un(t, h, v)− Un(t, π1(h), π2(v))‖L2(φδ) ≥ 2−1τ}+ Pr{sup|h|,|v|≤M ‖Un(t, π1(h), π2(v))− Un(t, 0, 0)‖L2(φδ) ≥ 2−1τ}.

Since sin and cos are Lipschitz functions with Lipschitz constant one, for each n with n−1/2M ≤2−1σ2,

|g(εn,j, t, h′A−1

n xn,j, (σ2 + n−1/2v)1/2)− g(εn,j, t, (π1(h))′A−1

n xn,j, (σ2 + n−1/2π2(v))1/2|

≤ 2|(σ2 + n−1/2v)−1/2t(εn,j − h′A−1/2n xn,j)

−(σ2 + n−1/2π2(v))−1/2t(εn,j − (π1(h))′A−1/2n xn,j)|

≤ 2|(σ2 + n−1/2v)−1/2t(εn,j − h′A−1/2n xn,j)− (σ2 + n−1/2π2(v))−1/2t(εn,j − h′A−1/2

n xn,j)|+2|(σ2 + n−1/2π2(v))−1/2t(εn,j − h′A−1/2

n xn,j)

−(σ2 + n−1/2π2(v))−1/2t(εn,j − (π1(h))′A−1/2n xn,j)|

≤ 4σ−3n−1/2η|t|(|εn,j|+ |h′A−1/2n xn,j|) + 4σ−1η|t||A−1/2

n xn,j|.

16

Hence, for each n large enough,

sup|h|,|v|≤M |Un(t, h, v)− Un(t, π1(h), π2(v))|≤ n−1/2

∑nj=1

(4σ−3n−1/2η|t||εn,j|+ 4σ−3n−1/2η|t|M |A−1/2

n xn,j|+ 4σ−1η|t||A−1/2n xn,j|

),

and

Pr{sup|h|,|v|≤M ‖Un(t, h, v)− Un(t, π1(h), π2(v))‖L2(φδ) ≥ 2−1τ} (4.25)

≤ 2τ−1E[sup|h|,|v|≤M ‖Un(t, h, v)− Un(t, π1(h), π2(v))‖L2(φδ)]

≤ 8τ−1σ−1η‖t‖L2(φδ)

(σ−2E[|εn,1|] + σ−2Mn−1

∑nj=1 |A−1/2

n xn,j|+ n−1/2∑n

j=1 |A−1/2n xn,j|

)

≤ 8τ−1σ−1η‖t‖L2(φδ)

(σ−2E[|εn,1|] + σ−2Mn−1/2p1/2 + p1/2

)

where we have used that by Lemma 4.1,

n−1/2∑n

j=1 |A−1/2n xn,j| ≤

(∑nj=1 |A−1/2

n xn,j|2)1/2

=(∑n

j=1 x′n,jA−1n xn,j

)1/2

= p1/2.

We claim that for each h, v,

‖Un(t, h, v)− Un(t, 0, 0)‖L2(φδ)Pr→ 0. (4.26)

We have that

‖Un(t, h, v)− Un(t, 0, 0)‖2L2(φδ) = n−1

n∑

j,k=1

∫

RYn,j(εn,j, t, h, v)Yn,k(εn,k, t, h, v)φδ(t) dt,

whereYn,j(x, t, h, v) := g(x, t, h′A−1/2

n xn,j, (σ2 + n−1/2v)1/2)− g(x, t, 0, σ)

−Gn(t, h′A−1/2n xn,j, (σ

2 + n−1/2v)1/2) + Gn(t, 0, σ).

Since sin and cos are Liptchitz functions, for each t, µ ∈ R and each s > 0,

|g(ε, t, µ, s)− g(ε, t, 0, σ)| ≤ 2|s−1t(ε− µ)− σ−1tε)| (4.27)

≤ 2|s−1t(ε− µ)− s−1tε|+ 2|s−1tε− σ−1tε| ≤ 2s−1|t||µ|+ 2s−1σ−1(s + σ)−1|t||ε||s2 − σ2|.

Using (4.27), we have that if n−1/2|v| ≤ 2−1σ2, then

E[n−1∑n

j=1

∫R(Yn,j(εn,j, t, h, v))2φδ(t) dt]

≤ E[n−1∑n

j=1

∫R(g(εn,j, t, h

′A−1/2n xn,j, (σ

2 + n−1/2v)1/2)− g(εn,j, t, 0, σ))2φδ(t) dt]

≤ E[n−1∑n

j=1

∫R

(8(σ2 + n−1/2v)−1t2(h′A−1/2

n xn,j)2

+8(σ2 + n−1/2v)−1σ−2((σ2 + n−1/2v)−1/2 + σ)−2t2ε2n,jn

−1v2)

φδ(t) dt

≤ E[n−1∑n

j=1

∫R

(16σ−2t2(h′A−1/2

n xn,j)2 + 6σ−6t2ε2

n,jn−1v2

)φδ(t) dt

= n−1σ−2(16|h|2 + 6σ−4v2E[ε2n,1])

∫R t2 φδ(t) dt,

17

E[n−1∑

1≤j<k≤n

∫R Yn,j(εn,j, t, h, v)Yn,k(εn,k, t, h, v)φδ(t) dt] = 0

and

Var(n−1

∑1≤j<k≤n

∫R Yn,j(εn,j, t, h, v)Yn,k(εn,k, t, h, v)φδ(t) dt

)

= n−2∑

1≤j<k≤n E[(∫R Yn,j(εn,j, t, h, v)Yn,k(εn,k, t, h, v)ϕδ(t) dt

)2]

≤ n−2∑

1≤j<k≤n

∫RE[(Yn,j(εn,j, t, h, v))2]E[(Yn,k(εn,k, t, h, v))2]φδ(t) dt]

≤ 2−1n−2∑n

j,k=1

∫RE[(Yn,j(εn,j, t, h, v))2]E[(Yn,k(εn,k, t, h, v))2]φδ(t) dt

≤ 2−1n−2∑n

j,k=1

∫RE[(g(εn,j, t, h

′A−1/2n xn,j, σ

2 + n−1/2v)− g(εn,j, t, 0, σ2))2]

×E[(g(εn,k, t, h′A−1/2

n xn,k, (σ2 + n−1/2v)1/2)− g(εn,k, t, 0, σ))2]φδ(t) dt

≤ 2−1n−2∑n

j,k=1

∫RE

[16σ−2t2(h′A−1/2

n xn,j)2 + 6σ−6t2ε2

n,jn−1v2

]

×E[16σ−2t2(h′A−1/2

n xn,k)2 + 6σ−6t2ε2

n,kn−1v2

]φδ(t) dt

≤ cn−2(∑n

j=1(h′A−1/2

n xn,j)2)2

+ cn−2∑n

j=1(h′A−1/2

n xn,j)2 + cn−2 = cn−2,

where we have used that

n∑j=1

(h′A−1/2n xn,j)

2 =n∑

j=1

h′A−1/2n xn,jx

′n,jA

−1/2n h = |h|2.

Hence, (4.26) follows. By (4.26),

Pr{ sup|h|,|v|≤M

‖Un(t, π1(h), π2(v))− Un(t, 0, 0)‖L2(φδ) ≥ 2−1τ} → 0. (4.28)

By (4.24), (4.25) and (4.28), for each η > 0,

lim supn→∞ Pr{sup|h|,|v|≤M ‖Un(t, h, v)− Un(t, 0, 0)‖L2(φδ) ≥ τ}≤ 8τ−1σ−1η‖t‖L2(φδ)(σ

−2E[|ε1|] + p1/2).

Since η > 0 is arbitrary the claim follows.

Proof of Theorem 2.3. It is easy to see that

‖n−1/2∑n

j=1(g(εn,j, t, 0, σ)− e−2−1t2(σ−1tεn,j − 2−1σ−2t2(ε2

n,j − σ2))) ‖2

L2(φδ)

= n−1∑n

j,k=1 h(σ−1εn,j, σ−1εn,k),

where

h(a, b) =

∫

R(cos(ta) + sin(ta)− exp(−2−1t2)

(1 + ta− 2−1t2(a2 − 1))

)(4.29)

×(cos(tb) + sin(tb)− exp(−2−1t2)(1 + tb− 2−1t2(b2 − 1))

)φδ(t) dt.

18

First, we prove that the functions h in (2.3) and in (4.29) agree. Using that φδ is an even

function,

∫R(cos(ta) + sin(ta)− exp(−2−1t2) (1 + ta− 2−1t2(a2 − 1)))

×(cos(tb) + sin(tb)− exp(−2−1t2) (1 + tb− 2−1t2(b2 − 1))) φδ(t) dt

=∫R(2

−1(1− i) exp(ita) + 2−1(1 + i) exp(−ita)− exp(−2−1t2) (1 + ta− 2−1t2(a2 − 1)))

×(2−1(1− i) exp(itb) + 2−1(1 + i) exp(−itb)− exp(−2−1t2) (1 + tb− 2−1t2(b2 − 1))) φδ(t) dt

=∫R exp(it(b− a))φδ(t) dt− ∫

R exp(ita)(1 + itb− 2−1t2(b2 − 1)) exp(−2−1t2)φδ(t) dt

− ∫R exp(itb)(1 + ita− 2−1t2(a2 − 1)) exp(−2−1t2)φδ(t) dt

+∫R(1 + ta− 2−1t2(a2 − 1))(1 + tb− 2−1t2(b2 − 1)) exp(−t2)φδ(t) dt

=∫R exp(it(b− a))φδ(t) dt

−(δ2 + 1)−1/2∫R exp(ita)(1 + tb− 2−1t2(b2 − 1))φδ(δ2+1)−1/2(t) dt

−(δ2 + 1)−1/2∫R exp(itb)(1 + ta− 2−1t2(a2 − 1))φδ(δ2+1)−1/2(t) dt

+(2δ2 + 1)−1/2∫R(1 + ta− 2−1t2(a2 − 1))(1 + tb− 2−1t2(b2 − 1))φδ(2δ2+1)−1/2(t) dt

= exp(−2−1δ2(a− b)2)

−(δ2 + 1)−1/2 exp(−2−1(δ2 + 1)−1δ2a2)

× (1 + (δ2 + 1)−1δ2b− 2−1(δ2 + 1)−2δ2(b2 − 1) (1− (δ2 + 1)−2δ2a2))

−(δ2 + 1)−1/2 exp(−2−1(δ2 + 1)−1δ2b2)

× (1 + (δ2 + 1)−1δ2a− 2−1(δ2 + 1)−2δ2(a2 − 1) (1− (δ2 + 1)−2δ2b2))

+(2δ2 + 1)−1/2 (1 + (2δ2 + 1)−1δ2(1− 2−1(a− b)2) + 2−23(2δ2 + 1)−2δ4(a2 − 1)(b2 − 1)) ,

where we have used that∫R exp(ita)φδ(t) dt = exp(−2−1a2δ2),∫R exp(ita)itφδ(t) dt = − exp(−2−1a2δ2)δ2a,∫R exp(ita)t2φδ(t) dt = exp(−2−1a2δ2)δ2(1− a2δ2),∫R t2φδ(t) dt = δ2 and

∫R t4φδ(t) dt = 3δ4.

Hence, the functions h in (2.3) and in (4.29) agree.

By the Gregory’s theorem for V–statistics over contiguous sequences,

n−1

n∑

j,k=1

h(σ−1εn,j, σ−1εn,k)

converges in distribution with the limit given by (2.5). Hence, to prove (2.6), it suffices to

prove that

n1/2D1/2n − ‖n−1/2

n∑j=1

(g(εn,j, t, 0, σ) + e−2−1t2(2−1σ−2t2(ε2

n,j − σ2) + σ−1tεn,j)) ‖L2(φδ) (4.30)

Pr→ 0.

19

It is easy to see that

Dn

= n−2∑n

j,k=1

∫R (cos(σ−1

n t(εn,j − εn,k))

−2 cos(σ−1n tεn,j) exp(−2−1t2) + exp(−t2)) φδ(t) dt

= n−2∑n

j,k=1

∫R(cos(σ−1

n tεn,j) + sin(σ−1n tεn,j)− exp(−2−1t2))

×(cos(σ−1n tεn,j) + sin(σ−1

n tεn,j)− exp(2−1t2))φδ(t) dt

= n−2∑n

j,k=1

∫R g(εn,j, t, εn,j − εn,j, σn)g(εn,k, t, εn,k − εn,k, σn)φδ(t) dt

= ‖n−1∑n

j=1 g(εn,j, t, (βn − β)′xn,j, σn)‖2L2(φδ).

Hence,

|n1/2D1/2n − ‖n−1/2

∑nj=1(g(εn,j, t, 0, σ) + e−2−1t2

(2−1σ−2t2(ε2

n,j − σ2) + σ−1tεn,j)) ‖L2(φδ)|

≤ ‖n−1/2∑n

j=1(g(εn,j, t, (βn − β)′xn,j, σn)− g(εn,j, t, 0, σ)

−Gn(t, (βn − β)′xn,j, σn) + Gn(t, 0, σ))‖L2(φδ)

+‖n−1/2∑n

j=1(Gn(t, (βn − β)′xn,j, σn)−Gn(t, 0, σ)

− exp(−2−1t2)(2−1σ−2t2(σ2n − σ2)− σ−1t(βn − β)′xn,j))‖L2(φδ)

+‖e−2−1t22−1σ−2t2(n1/2(σ2n − σ2)− n−1/2

∑nj=1(ε

2n,j − σ2))‖L2(φδ)

+‖n−1/2∑n

j=1 σ−1t exp(−2−1t2)((βn − β)′xn,j − εn,j

)‖L2(φδ))

=: I + II + III + IV.

Hence, to prove (4.30), it suffices to show that I, II, III, IVPr→ 0.

By Lemma 4.3, for each 0 < M < ∞,

sup|h|≤M,|v|≤M

‖n−1/2

n∑j=1

(g(εn,j, t, h′A−1/2

n xn,j, (σ2 + n−1/2v)1/2) (4.31)

−Gn(t, h′A−1/2n xn,j, (σ

2 + n−1/2v)1/2)− g(εn,j, t, 0, σ) + Gn(t, 0, σ))‖L2(φδ)Pr→ 0.

Plugging (A1/2n (βn − β), n1/2(σ2

n − σ2)) as (h, v) in (4.31), we get that

IPr→ 0. (4.32)

Notice that by Lemma 4.2, (A1/2n (βn − β), n1/2(σ2

n − σ2)) = OP (1).

We have that

Gn(t, µ, s) = G(t, µ, s) + n−1/2

∫

R(cos(s−1t(y − µ)) + sin(s−1t(y − µ)))αn(y)φσ(y) dy,

where G(t, µ, s) = E[g(ε1, t, µ, s)]. To prove that IIPr→ 0, we show that

‖n−1/2∑n

j=1(G(t, (βn − β)′xn,j, σn)−G(t, 0, σ) (4.33)

− exp(−2−1t2)(2−1σ−2t2(σ2n − σ2)− σ−1t(βn − β)′xn,j))‖L2(φδ)

Pr→ 0

20

and

‖n−1∑n

j=1(∫R(cos(σ−1

n t(y − (βn − β)′xn,j)) + sin(σ−1n t(y − (βn − β)′xn,j)) (4.34)

− cos(σ−1ty)− sin(σ−1ty))αn(y)φσ(y) dy‖L2(φδ)Pr→ 0.

SinceE[exp(is−1t(ε1 − µ)] = exp(−is−1tµ− 2−1t2s−2σ2)

= exp(−2−1t2s−2σ2)(cos(s−1tµ)− i sin(s−1tµ)),

G(t, µ, s) = exp(−2−1t2s−2σ2)(cos(s−1tµ)− sin(s−1tµ))− exp(−2−1t2).

Hence,

G(t, µ, s)−G(t, 0, σ)− exp(−2−1t2)(2−1σ−2t2(s2 − σ2)− σ−1tµ) (4.35)

= exp(−2−1t2s−2σ2)(cos(s−1tµ)− sin(s−1tµ))

− exp(−2−1t2)− exp(−2−1t2)(2−1σ−2t2(s2 − σ2)− σ−1tµ)

= exp(−2−1t2)(exp(−2−1t2(s−2σ2 − 1))− 1 + 2−1t2(s−2σ2 − 1))(cos(s−1tµ)− sin(s−1tµ))

+ exp(−2−1t2)(1− 2−1t2(s−2σ2 − 1))(cos(s−1tµ)− sin(s−1tµ)− 1 + s−1tµ)

+ exp(−2−1t2)2−1t2(−(s−2σ2 − 1)− σ−2(s2 − σ2))

+ exp(−2−1t2)(s−1tµ− σ−1tµ)

+ exp(−2−1t2)2−1t2(s−2σ2 − 1)s−1tµ

=: V + V I + V II + V III + IX.

Using that for each x ∈ R, |ex − 1− x| ≤ 2−1x2e|x|,

|V | ≤ exp(−2−1t2 + 2−1t2|s−2σ2 − 1|)2−2t4s−4(s2 − σ2)2.

We also have that

|V I| ≤ a exp(−2−1t2)(1 + 2−1t2s−2|s2 − σ2|)µ2,

|V II| ≤ 2 exp(−2−1t2)2−1t2|s2 − σ2||s−2 − σ−2| = exp(−2−1t2)t2s−2σ−2(s2 − σ2)2,

|V III| ≤ 2−1 exp(−2−1t2)|t|3s−3|s2 − σ2|µ|,where a := supx 6=0 x−2| cos x− 1|+ supx6=0 x−2| sin x− x|. Hence,

|G(t, µ, s)−G(t, 0, σ)− exp(−2−1t2)(2−1σ−2(s2 − σ2)− σ−1tµ)| (4.36)

≤ exp(−2−1t2 + 2−1t2s−2|s2 − σ2|)2−2t4s−4(s2 − σ2)2

+a exp(−2−1t2)(1 + 2−1t2)s−2|s2 − σ2|µ2,

+ exp(−2−1t2)t2s−2σ−2(s2 − σ2)2

+2−1 exp(−2−1t2)|t|3s−3|s2 − σ2|µ|

21

and

‖n−1/2∑n

j=1(G(t, (βn − β)′xn,j, σn)−G(t, 0, σ)

− exp(−2−1t2)(2−1σ−2t2(σ2n − σ2)− σ−1t(βn − β)′xn,j))‖L2(φδ) (4.37)

≤ 2−2‖ exp(−2−1t2 + 2−1t2σ−2n |σ2

n − σ2|)t4‖L2(φδ)σ−4n n1/2(σ2

n − σ2)2

+a(‖ exp(−2−1t2)‖L2(φδ) + 2−1‖ exp(−2−1t2)t2‖L2(φδ)σ

−2n |σ2

n − σ2|) n−1/2∑n

j=1((βn − β)′xn,j)2

+‖ exp(−2−1t2)t2‖L2(φδ)σ−2n σ−2(σ2

n − σ2)2

+‖ exp(−2−1t2)|t|3‖L2(φδ)σ−3n |σ2

n − σ2|n−1/2∑n

j=1 |(βn − β)′xn,j|,

which tends in probability to zero, because, by Lemma 4.2, n1/2(σ2n − σ2) = OP (1),

n∑j=1

((βn − β)′xn,j)2 =

n∑j=1

(βn − β)′xn,jx′n,j(βn − β) = (βn − β)′An(βn − β) = OP (1)

and

n−1/2

n∑j=1

|(βn − β)′xn,j| ≤(

n∑j=1

((βn − β)′xn,j)2

)1/2

= OP (1).

Therefore, (4.33) follows.

For each a, h ∈ R, we have that

| sin(a + h)− sin(a)− h cos(a)|≤ | sin(a) cos(h)− sin(a)|+ | cos(a) sin(h)− h cos(a)| ≤ ch2

and| cos(a + h)− cos(a) + h sin(a)|

≤ | cos(a) cos(h)− cos(a)|+ | sin(a) sin(h)− h sin(a)| ≤ ch2.

So,

‖n−1∑n

j=1(∫R(cos(σ−1

n t(y − (βn − β)′xn,j)) + sin(σ−1n t(y − (βn − β)′xn,j))

− cos(σ−1ty)− sin(σ−1ty))αn(y)φσ(y) dy‖L2(φδ)

≤ ‖n−1∑n

j=1

∫R(cos(σ−1ty)− sin(σ−1ty))

×(σ−1

n t(y − (βn − β)′xn,j)− σ−1ty)

αn(y)φσ(y) dy‖L2(φδ)

+c‖n−1∑n

j=1

∫R

(σ−1

n t(y − (βn − β)′xn,j)− σ−1ty)2

αn(y)φσ(y) dy‖L2(φδ)

≤ ∫R |y||αn(y)|φσ(y) dy‖t‖L2(φδ)|σ−1

n − σ−1|+

∫R |αn(y)|φσ(y) dy‖t‖L2(φδ)|σ−1

n ||n−1∑n

j=1(βn − β)′xn,j|+c

∫R y2αn(y)φσ(y) dy‖t2‖L2(φδ)(σ

−1n − σ−1)2

+c∫R |αn(y)|φσ(y) dy‖t2‖L2(φδ)σ

−2n n−1

∑nj=1((βn − β)′xn,j)

2

Pr→ 0,

22

because for k = 0, 1, 2,∫R |y|k|αn(y)|φσ(y) dy

≤ ∫R |y|k|α(y)|φσ(y) dy +

∫R |y|k|αn(y)− α(y)|φσ(y) dy

≤ (∫R(α(y))2φσ(y) dy

∫R y2kφσ(y) dy

)1/2+

(∫R(αn(y)− α(y))2φσ(y) dy

∫R y2kφσ(y) dy

)1/2

= 0(1),

n−1∑n

j=1((βn − β)′xn,j)2 = n−1(βn − β)′An(βn − β)

Pr→ 0

and

n−1

n∑j=1

(βn − β)′xn,jPr→ 0. (4.38)

By (1.4),

n−1/2

n∑j=1

(βn − β)′xn,j = n−1/2

n∑

j,k=1

x′n,kA−1n xn,jεn,k.

We have that

E[n−1/2∑n

j,k=1 x′n,kA−1n xn,jεn,k − n−1/2

∑nk=1 εn,k] (4.39)

=(n−1

∑nj,k=1 x′n,kA

−1n xn,j − 1

)n1/2E[εn,1] → 0.

and

Var(n−1/2∑n

j,k=1 x′n,kA−1n xn,jεn,k − n−1/2

∑nk=1 εn,k) (4.40)

= (σ2 + o(1))n−1∑n

k=1

(∑nj=1 x′n,kA

−1n xn,j − 1

)2

= (σ2 + o(1))

(n−1

∑nk=1

(∑nj=1 x′n,kA

−1n xn,j

)2

− 2σ2n−1∑n

k=1

∑nj=1 x′n,kA

−1n xn,j + 1

)

= (σ2 + o(1))(n−1

∑nk=1

∑nj=1

∑nl=1 x′n,jA

−1n xn,kx

′n,kA

−1n xn,l − 2n−1

∑nk=1

∑nj=1 x′n,kA

−1n xn,j + 1

)

= (σ2 + o(1))(1− n−1

∑nk=1

∑nj=1 x′n,kA

−1n xn,j

)→ 0,

using condition (B). Hence, (4.38) follows. Therefore, (4.34) follows.

By Lemma 4.2,

III = ‖e−2−1t22−1σ−2t2(n1/2(σ2n − σ2)− n−1/2

∑nj=1(ε

2n,j − σ2))‖L2(φδ) (4.41)

≤ |n1/2(σ2n − σ2)− n−1/2

∑nj=1(ε

2n,j − σ2)|‖e−2−1t22−1σ−2t2‖L2(φδ)

Pr→ 0.

By (4.39) and (4.40),

IV ≤ ‖n−1/2∑n

j=1 σ−1t exp(−2−1t2)((βn − β)′xn,j − εn,j)‖L2(φδ) (4.42)

≤ |n−1/2∑n

j=1((βn − β)′xn,j − εn,j)|‖σ−1t exp(−2−1t2)‖L2(φδ)Pr→ 0.

Therefore, (4.30) follows from (4.32)–(4.34), (4.41) and (4.42). ¤

23

References[1] Baringhaus, L. and Henze, N. (1988). A consistent test for multivariate normality based on the

empirical characteristic function. Metrika 35 339–348.[2] de la Pena, V. H. and Gine, E. (1999). Decoupling. From Dependence to Independence. Springer–

Verlag, New York.[3] Epps, T. W. and Pulley, L. B. (1983). A test for normality based on the empirical characteristic

function. Biometrika 70 723–726.[4] Gregory, G. G. (1977). Large sample theory for U–statistics and tests of fit. Ann. Statist. 5

110–123.[5] Hajek, J. and Sidak, Z. (1967). Theory of Rank Tests. Academic Press, New York.[6] Henze, N. (2002). Invariant tests for multivariate normality: a critical review. Statist. Pap. 43

467–506.[7] Henze, N. and Zirkler, B. (1990). A class of invariant consistent tests for multivariate normality.

Comm. Statist. Theory Methods 19 3595–3617.[8] Jureckova, J.; Picek, J. and Sen, P. K. (2003). Goodness–of–fit test with nuisance regression

and scale. Metrika 58 235–258.[9] Lee, A. J. (1990). U–statistics, Theory and Practice. Marcel Dekker, Inc., New York.

[10] Mecklin, C. J. and Mundfrom, D. J. (2004). An appraisal and bibliography of tests for multi-variate normality. Internat. Statist. Rev. 72 123-138.

[11] Neter, J.; Wasserman, W.; Kutner, M. H.; Nachtsheim, C.; and Nachtsheim, C. J. (1996).Applied Linear Statistical Models 4–th Edition. McGraw–Hill, New York.

[12] Sen, P. K.; Jureckova, J.; and Picek, J. (2003). Goodness–of–fit test of Shapiro–Wilk type withnuisance regression and scale. Austrian J. of Statist. 32 No 1 & 2 163-177.

[13] Shapiro, S. S. and Wilk, M. B. (1965). An analysis of variance test for normality: completesamples. Biometrika 52 591–611.

[14] Shapiro, S. S. and Wilk, M. B. (1968). Approximations for the null distribution of the Wstatistic. Technometrics 10 861–866.

24