hypothesis testing with communication constraintsdineshk/eecs750_presentation.pdf · 2006. 4....

Hypothesis Testing with Communication Constraints

Dinesh Krithivasan

EECS 750

April 17, 2006

Dinesh Krithivasan (EECS 750) Hyp. testing with comm. constraints April 17, 2006 1 / 21

Presentation Outline

1 Hypothesis TestingStein’s Lemma

2 Communication ConstraintsProblem FormulationGeneral Bivariate Hypothesis TestingA Single-Letter Lower BoundSpecial Cases

3 Conclusions and References


Outline





Bivariate Hypothesis Testing

Given sensor measurements x1, x2, . . . , xn, determine if an earthquakeoccurred or not?

X1,X2, . . . ,Xn be i.i.d

Hypothesis H0 : Distribution is P(x)Hypothesis H1 : Distribution is Q(x)

Statistician’s task : Decide on H0 or H1 based on xn = x1, x2, . . . , xn

Decision rule : Declare H0 if xn ∈ A ⊂ X n, else declare H1


Error events

Two kinds of errors

False Alarm: Declare H0 as H1.Miss: Declare H1 as H0.

Corresponding probabilities

P(Error of type 1) , α = Pn(Ac)P(Error of type 2) , β = Qn(A)

Usually there is a trade-off between α and β


Stein’s Lemma

Let α go to 0 arbitrarily slowly with n

What is the best we can do as regards the probability of type 2 error β?

Answer given by Stein’s lemma

Stein’s Lemma

limn→∞

1

nlog βn(ε) , θ(ε) = −D(P ‖ Q) ∀ε ∈ (0, 1)

Can be proved by using the typical set as the acceptance region


Outline





Problem Formulation

Common assumption is that all data is known to the statistician inadvance

What if he/she can be informed about the data at a finite rate R?

Not a significant constraint if data is collected at a single location or ifonly one random variable is present

In the above case, transmission of one bit is sufficient to enableoptimal decision

Problem is interesting when different variables are measured atdifferent locations


Problem formulation

X

Y

X Encoder

Y Encoder

StatisticianH

f( Xn

n

n)

g( Yn)

0

H1

This notion of encoding is more general than standard source coding


General Bivariate Hypothesis Testing

Hypothesis testing with 2 arbitrary Hypothesis PXY (Hypothesis H0)and PX Y (Hypothesis H1)

Statistician observes X n and Y n via encoding functions of rate (R1, R2)

We are interested in asymptotics of θ(R1,R2)(n, ε)

Assume for simplicity that R2 = ∞Will derive an achievable lower bound θL(R, ε) to θ(R, ε)


Key Ideas

Choose acceptance region to be the ”typical set” under hypothesis H0

Decoder has access only to the types Pun , Pyn and Pun,yn

Decoder reproduces the largest family of random variables it can usingavailable information

Exponent will be the divergence between the families resulting from H0

and H1

Larger the family, larger the divergence since

D(X1Y1 ‖ X2Y2) ≥ D(X1 ‖ X2)


Lemma

We need the following lemma

Let U,X ,Y be finite random variables such that U − X − Y . Then thereexists u1, . . . , uM ∈ T n

µ (U) (M = exp[n(I (U;X ) + η)]) and M disjointsubsets C1, . . . , CM ∈ T n

µ (X | ui ) for which

M∑i=1

{P(X nY n ∈ Ci × T nµ (Y |ui ))} ≥ 1− δ

for any fixed η > 0 and δ > 0

Proof using standard information-theoretic ideas


Hypothesis Testing Scheme

Let M, ui , Ci be as given in the lemma

X -encoder defined as

f (xn) =

{i if xn ∈ Ci

0 else

Statistician has access to i ∈ {1, 2, . . . ,M} and yn

Decision Rule : Declare H0 if yn ∈ T nµ (Y | ui )


Acceptance Region

Decision rule induces an acceptance region An given by

An =M⋃i=1

(Ci × T nµ (Y | ui ))

No single module has all the information required to determine if(xn, yn) ∈ An

Probability of type-1 error is bounded by the lemma

αn = P(X nY n ∈ Acn) ≤ δ

Need to bound βn =∑

(xn,yn)∈AnP((X nY n) = (xn, yn))

Can be done using type-counting in An


An Achievable Lower Bound

Two sets of Auxiliary random variables

S(R) = {U : I (U;X ) ≤ R,U − X − Y }

L(U) = {UX Y : P(UX ) = P(UX ),P(UY ) = P(UY )}

Define U to satisfy U − X − Y and P(U | X ) = P(U | X )

Describes the X -Encoder. Rate constraint isautomatically met

For every R ≥ 0 and 0 < ε < 1, the exponent

θL(R, ε) = supU∈S(R)

minUX Y∈L(U)

D(UX Y ‖ UX Y )

is achievable




S(R) = {U : I (U;X ) ≤ R,U − X − Y }L(U) = {UX Y : P(UX ) = P(UX ),P(UY ) = P(UY )}


Decoder can reproduce the set of jointtypes P(un,yn) and P(un,xn)



minUX Y∈L(U)

D(UX Y ‖ UX Y )

is achievable






Same Encoder used in case of eitherhypothesis



minUX Y∈L(U)

D(UX Y ‖ UX Y )

is achievable








minUX Y∈L(U)

D(UX Y ‖ UX Y )

is achievable


Special Case: Lower Bound of Ahlswede and Csiszar

For any U ∈ S(R), we have

θL(R, ε) ≥ D(X ‖ X ) + D(UY ‖ UY )

where Y is such that U − X − Y and P(Y | X ) = P(Y | X )

Follows from simple algebraic manipulationsThis lower bound doesn’t exploit P(un,xn) and is consequently weaker


Special Case: Test against Independence

Suppose P(X Y ) = P(X )P(Y ). Then for any 0 < ε < 1,

θL(R, ε) ≥ maxU∈S(R)

I (U;Y )

Follows from simple algebraic manipulationsThis case was completely solved by Ahlswede and Csiszar who provedthe converse as wellTheir proof used Divergence-Characterization techniques


Further Comments

If R ≥ H(X ), then the lower bound becomes θL(R, ε) = D(XY ‖ X Y )

Extension to two sided compression is straight-forward

Involves introduction of further auxiliary random variables V and V

Approach seems best suited to get achievability results

Divergence characterization techniques better suited for converses

Lower bound can be significantly tightened

Above encoding scheme for zero-error reconstruction of the joint typesCan consider encoders that reconstructions with exponentially low errorprobability


Outline





Conclusions

Bivariate hypothesis testing with one sided data compression wasstudied

A single-letter lower bound to the power exponent was derived

This bound subsumes other known bounds and achievability results

Easily extendable to two-sided compression case

Other statistical problems such as parameter estimation and patternclassification also studied under similar rate constraints


References

R.Ahlswede and I.Csiszar, ”Hypothesis Testing with Communicationconstraints”, IEEE trans. on info. theory, vol. IT-32, No.4 July 1986

Te Sun Han, ”Hypothesis Testing with Multiterminal DataCompression”, IEEE trans. on info. theory, vol. IT-33, No.6 November1987

Te Sun Han and Shun-ichi Amari, ”Statistical Inference UnderMultiterminal Data Compression”, IEEE trans. on info. theory, vol. 44,No.6 October 1998

R.Ahlswede and Janos Korner, ”Source coding with side informationand a converse for degraded broadcast channels”, IEEE trans. on info.theory, vol. IT-21, No.6 November 1975


hypothesis testing with communication constraintsdineshk/eecs750_presentation.pdf · 2006. 4....

Documents