on hierarchical type covering ertem tuncel 1, jayanth nayak 2, and kenneth rose 2 1 university of...

On Hierarchical Type Covering

Ertem Tuncel1, Jayanth Nayak2, and Kenneth Rose2

1 University of California, Riverside2 University of California, Santa Barbara

TUNCEL, NAYAK, AND ROSE: ON HIERARCHICAL TYPE COVERING

Types and Typical Sequences

Type class : Collection of data points with empirical distribution P, i.e.,

nPT x

)()|(1

aPaNn

x

Conditional type class (a.k.a. V-shell): Collection of data points with empirical conditional distribution V(x|y) w.r.t. , i.e.,

)(ynVT

)|()|(1

),|,(1

baVbNn

baNn

yyx

xy


Type Covering Covering of using “distortion spheres” about

codevectors , i = 1, 2,…, M, i.e.,

nPT

)(iy

1( ( ), )

Mn

Pi

T S i D

y

where DidDiS ))(,(:)),(( yxx y

Type covering lemma (Csiszár and Körner) states that there exists a covering strategy with

)()(log1

noDRMn P


Uses of Type Covering

)(DRP

),( RDEP

Source coding: Proof of achievability of the rate-distortion curve Proof of universal achievability of Marton’s error exponent

in lossy source coding Results on rate redundancy

Guessing subject to distortion: Alice presents Bob fixed guesses until

Minimum guessing exponent is achieved

by the concatenation of type-covering codebooks in acertain order.

),...2(),1( yy Did ))(,( yx

})({log1 XiEn


2-Stage Type Covering

Weak covering: Implicit in Tuncel and Rose’s work on

error exponents in scalable source coding, is the building

of a tree of spheres and such

that for each , at least one parent-child pair (i, j)

simultaneously covers .

)),(( 11 DiS y )),|(( 22 DijS y

xx

Two possibilities: Strong covering: Introduced by Kanlis and Narayan, is the

covering of using spheres ,

followed by a covering of each

using spheres , .

nPT )),(( 11 DiS y 11 Mi

)),(( 11 DiST nP y

)),|(( 22 DijS y 21 Mj


Illustration of the Difference

nPT

STRONG COVERING WEAK COVERING

Every x is covered by at least one parent-child

pair

nPT


Need for Strong vs. Weak The difference was previously overlooked. Even though strong covering was introduced in the

context of scalable source coding, weak covering suffices for that purpose.

),2(),1( 11 yy 111 ))(,( Did yx

?})]()({[log1

XX jiEn

),|2(),|1( 22 ii yy 222 ))|(,( Dijd yx

Strong covering, on the other hand, is useful in the generalization of the guessing game, introduced by Merhav, Arikan, and Roth: Alice presents guesses until

and then guesses until

Minimum guessing exponent


Main Results We derive necessary and sufficient conditions for

achievability of covering rates and

for both strategies.

1log1

Mn 2log

1M

n

The claimed rate region in Kanlis and Narayan’s work for strong covering is in fact the rate region for weak covering.

All the type covering based results regarding error exponents remain valid, as weak covering is sufficient for those analyses.

Strong covering rates can be strictly higher than weak covering rates (shown by example).

The minimum achievable hierarchical guessing exponent result must be re-examined, as it relies on strong covering.


Let R P(D1,D2) denote the region of all (R1,R2)

satisfying the following:

Q1(y1), Q2|1(y2|y1), V(x|y1), and W(x| y1, y2) such that

V V (P,Q1,D1), i.e., V and Q1 are consistent with P and induce distortion D1,

WW (V,Q2|1,Q1,D2), i.e., W and Q2|1 are consistent with V, and (together with Q1) induce distortion D2,

Weak Covering Rates

1 1 1

1 2 1 1 2|1 1 2

; ( , )

; ; ( , ) ( , | )

I X Y I Q V R

I X Y I X Y Y I Q V I Q W Q R


Weak Covering Rates If (R1,R2) R P(D1,D2), then there exists a weak

covering with

2211

111

log

log

RMM

RM

n

n

Conversely, if there exists a weak covering with codebook sizes M1 and M2, then

),( )log ,log( 21211

11 DDMMM Pnn R


Strong Covering Rates Let TP(D1,D2) denote the region of all (R1,R2) with

R2 R1 satisfying the following:

Q1(y1) such that

,

For all V V (P,Q1,D1), Q2|1(y2|y1) such that

Note that TP(D1,D2) R P(D1,D2)


Strong Covering Rates If (R1,R2) TP(D1,D2) , then there exists a strong

covering with

2211

111

log

log

RMM

RM

n

n

Conversely, if there exists a strong covering with codebook sizes M1 and M2, then

),( )2log ,log( 21211

11 DDMMM Pnn T


Example of TP (D1,D2)R P

(D1,D2)

),( yxd

011.02

1.0011

11.000

210\ yx 3/1)( xP

06.01 D

016.02 D

When R1= RP(D1) , there exists a unique pair (Q1,V)

such that V V (P,Q1,D1) and I(Q1,V) R1

)|,(min)|,(min 11|2),,',( ,

11|2),,,( , 211|21|2211|21|2

QWQIQWQIDQQVWQDQQVWQ WW

Then it suffices to find VV (P,Q1,D1) such that

We found such V


Consequences on Guessing

Merhav, Arikan, and Roth proved the following lower bound for the minimum guessing exponent

where

)||'()',,(max})]()({[log1

inflim 21'

PPPDDKjiEn Pn

D

XX

)|,(),,( maxmin),,( 11|21

),,,(),,(

,21

211|2

11

,1|21

QWQIVQIPDDK

DQQVWDQPV

QQ

WV

Using strong type covering, they also proved achievability of this lower bound when Alice knows the type of x in advance.


Consequences on Guessing

Proof assumes every rate pair in R P (D1,D2) is sufficient for covering.

What is achievable, instead, is

where

)||'()',,(max 21'

PPPDDLP

D

)|,(minmax),,( min maxmin),,( 11|2

),,,(,),,(

1),,(

21

211|2

1|211111

QWQIVQIPDDLDQQVW

QDQPVDQPVQW

VV

For the example yielding TP (D1,D2) R P (D1,D2), we also show that L(D1,D2,P) > K(D1,D2,P) .


To sum up… Contrasted two possibilities for hierarchical type

covering. Presented rate regions for both cases.

Rate regions are not identical. Investigated the effect of this distinction on the

guessing problem.

on hierarchical type covering ertem tuncel 1, jayanth nayak 2, and kenneth rose 2 1 university of...

Documents