data-driven goodness-of-fit tests · data-driven goodness-of-ﬁt tests mikhail langovoy institute...

Data-driven goodness-of-fit tests

Mikhail Langovoy

Institute for Applied Mathematics,University of Bonn

September 1, 2008

M. Langovoy (University of Bonn) Data-driven tests 01.09.2008 1 / 34

Outline

1 IntroductionMain ideasData-driven tests

2 General notionsSelection ruleFramework

3 NT-statisticsDefinitionsExamplesTheorems

Outline

Two Approaches

Constructing good tests is an essential problem of statistics.

Two approaches:

direct: "distance" between theoretical and empirical distribution isproposed as statistic

aiming optimality: construct tests which are asymptotically efficient(Neyman, Le Cam, Wald)

Two Approaches

Constructing good tests is an essential problem of statistics.

Two approaches:

direct: "distance" between theoretical and empirical distribution isproposed as statistic

aiming optimality: construct tests which are asymptotically efficient(Neyman, Le Cam, Wald)

Examples of Distance-Based Statistics

Kolmogorov - Smirnov:

Dn =√

n∥∥Fn − F

∥∥∞

Cramer - von Mises:

ω2n = n

∫ ∞−∞

(Fn(t)− F0(t)

)2 d F0(t)

many other

About Distance-Based Tests

These tests works

asymptotically optimal only in a few directions of alternatives

Another type: Neyman’s Statistic

hypothesis H0 : X ∼ U[0,1]

{φj}∞j=0 orthonormal basis of L2([0,1], λ)

Nk =k∑

n∑i=1

φj(Xi)

Nk =k∑

n∑i=1

φj(Xi)

Nk =k∑

n∑i=1

φj(Xi)

Outline

Idea of Selection Rule

model dimension k was known fixed in advance

important: select the right model dimension!

incorrect choice can decrease the power of a test

Solution: incorporate the test statistic by some procedurechoosing the right dimension automatically by the data

Bonuses of data-driven score tests

Data-driven score tests are

asymptotically optimal in an infinite number of directions

show good overall performance in practice

Outline

Penalty

Definitionnested family of models Mk for k = 1, . . . , d(n)

d(n) control sequence

choose π(·, ·) : N× N→ R

assume

π(1,n) < π(2,n) < . . . < π(d(n),n) for all n

π(j ,n)− π(1,n)→∞ as n→∞ for j = 2, . . . ,d(n)

Call π(j ,n) a penalty attributed to model Mj and sample size n.

Penalty

assume

π(j ,n)− π(1,n)→∞ as n→∞ for j = 2, . . . ,d(n)

Penalty

assume

π(j ,n)− π(1,n)→∞ as n→∞ for j = 2, . . . ,d(n)

Penalty

assume

π(j ,n)− π(1,n)→∞ as n→∞ for j = 2, . . . ,d(n)

Selection rule

Tk : testing validity of Mk

DefinitionA selection rule S for the sequence of statistics {Tk} is

S = min{

k : 1 ≤ k ≤ d(n); Tk − π(k ,n) ≥ Tj − π(j ,n), j = 1, . . . ,d(n)}

Call TS a data-driven test statistic for testing validity of the initial model.

Outline

Framework

X1,X2, . . . random variables, values in a measurable space X

X1, . . . ,Xm have joint distribution Pm ∈ Pm - for every m

function F acting from ⊗∞m=1 Pm = (P1,P2, . . .) to a known set Θ

F(P1,P2, . . .) = θ

Framework

X1,X2, . . . random variables, values in a measurable space X

X1, . . . ,Xm have joint distribution Pm ∈ Pm - for every m

function F acting from ⊗∞m=1 Pm = (P1,P2, . . .) to a known set Θ

F(P1,P2, . . .) = θ

Framework

H0 : θ ∈ Θ0 ⊂ Θ

HA : θ ∈ Θ1 = Θ \Θ0

observations Y1, . . . , Yn, values in a measurable space Y

not necessarily on the basis of X1, . . . , Xm !

Framework

H0 : θ ∈ Θ0 ⊂ Θ

HA : θ ∈ Θ1 = Θ \Θ0

not necessarily on the basis of X1, . . . , Xm !

Outline

Main concept

Definition

k fixed number

l = (l1, . . . , lk ) vector-function

li : Y→ R for i = 1, . . . , k are known Lebesgue measurable

Main concept

Definition

L = {E0[l(Y )]T l(Y )}−1

E0 is with respect to P0

P0 is the d.f. of some (fixed and known) random variable Y

Y has values in Y

assume

E0 l(Y ) = 0

L is well defined

Main concept

Definition

L = {E0[l(Y )]T l(Y )}−1

E0 is with respect to P0

P0 is the d.f. of some (fixed and known) random variable Y

Y has values in Y

assume

E0 l(Y ) = 0

L is well defined

Main concept

Definition

{1√n

n∑j=1

Tk - statistic of Neyman’s type (NT-statistic)

Outline

New possibilities

l1, . . . , lk can be

some score functions

any other functions, depending on the problem

truncated, penalized or partial likelihood

possible to use l1, . . . , lk unrelated to any likelihood

New possibilities

l1, . . . , lk can be

New possibilities

l1, . . . , lk can be

Important applications

statistical inverse problems

rank tests for independence

semiparametric regression

Statistical inverse problems

Deconvolution

applications from signal processing to psychology

basic statistical inverse problem

Deconvolution

instead of Xi one observes Yi

Yi = Xi + εi

ε′is are i.i.d. with a known density h

Xi and εi are independent for each i

H0 : X has density f0

Deconvolution

instead of Xi one observes Yi

Yi = Xi + εi

ε′is are i.i.d. with a known density h

Xi and εi are independent for each i

H0 : X has density f0

Deconvolution

choose for every k ≤ d(n) an auxiliary parametric family {fθ}

θ ∈ Θ ⊆ Rk

f0 from this family coincides with f0 from the null hypothesis H0

the true F possibly has no relation to the chosen {fθ}

Deconvolution

l(y) =

∂∂θ

( ∫R fθ(s) h( y − s) ds

)∣∣∣θ=0∫

R f0(s) h( y − s) ds

define Tk as above ⇒ Tk is an NT-statistic

Outline

Null distribution

Theorem{Tk} sequence of NT-statistics

S selection rule, with penalty of proper weight

large deviations of Tk are properly majorated

d(n) ≤ min{un ,mn} .

Then S = OP0(1) and TS = OP0(1).

Consistency condition

〈C〉 there exists integer K = K (P) ≥ 1 such that

EP l1(Y ) = 0, . . . ,EP lK−1(Y ) = 0, EP lK = CP 6= 0

Consistency theorem

Theorem{Tk} sequence of NT-statistics

S selection rule, with penalty of proper weight

the regularity assumptions are satisfied

d(n) = o(rn), d(n) ≤ min{un,mn}.

Then TS is consistent against any (fixed) alternative P satisfying (C).

data-driven goodness-of-fit tests · data-driven goodness-of-ﬁt tests mikhail langovoy institute...

Documents

goodness of fit tests for categorical data: comparing stata,...

goodness of fit tests: independence

11.1 name: chi-square tests for goodness of fit

ap stats ~ lesson 11a: chi-square goodness of fit tests ·...

new goodness-of-fit tests for pareto … · tests for...

entropy statistics tests goodness of fit wald lagrange...

history and modern developments in goodness of fit tests -...

evaluation of goodness-of-fit tests for the cox

conditional tests of goodness of fit

1 1 slide © 2005 thomson/south-western chapter 12 tests of...

fast large-sample goodness-of-fit tests for copulas -...

fast large-sample goodness-of- t tests for...

e cient goodness-of- t tests in multi-dimensional vine ......

qca and fuzzy set goodness of-fit tests by wendy...

composite goodness-of-fit tests for left-truncated loss...

chi-square goodness of fit tests chi-square goodness of fit

14 goodness-of-fit tests and categorical data analysis

comparison of the goodness-of-fit tests for truncated

goodness-of-fit tests in reliability: weibull distribution

goodness of fit tests - math.uh.edu