communicating correlated sources over user...

Communicating Correlated Sources over 2−user MAC

Arun Padakandla

Center for Science of Information, Purdue University

June 28, 2017

Communicating distributed correlated sources over channelsPart 2 : Joint source-channel coding technique

Arun Padakandla

Center for Science of Information, Purdue University

June 28, 2017

Problem Setup : Transmitting Correlated sources over 2−user MAC

S1S2X2

YWY|X1X2

Shannon-theoretic study

?? Optimal coding tech. ??

?? Necessary and sufficient conditions in terms of WS1S2 , WY |X1X2. ??

Plain Vanilla Lossless source-channel coding over 2-user MAC.

1. A new coding tech.

2. Characterize performance, derive new sufficient conditions.

S1S2X2

YWY|X1X2

S1S2X2

YWY|X1X2

Big picture : Where is this going?

General multi-terminal problems : Seems NO single-letter (S-L) expr. forcapacity.

Example : Our fauvorite point-to-point chnl (PTP).

Capacity [Shannon ’48] = suppX

I(X;Y ), and NOT (1)

Capacity [Shannon ’48] = supn≥1

suppnX

nI(Xn;Y n)

suppXand supn≥1 suppn

X: world of difference .

Associated with (1) is a single-letter (S-L) coding tech.

suppnX

nI(Xn;Y n)

suppnX

nI(Xn;Y n)

What if this were not true? A general multi-terminal scenario

Capacity > suppX|S

α(pS1···SlX1···XmY1···Ym) for every functional α(·)

Admits no optimal S-L coding techn.

This work addresses the above difficulty.

Contribution : S-L expr. for multi-letter (M-L) coding techs.

A multi-letter (M-L) coding tech. by carefully stitching together S-L techs.

Characterize (inner bound) ‘performance’ via S-L expr.Yes suppX

α(pXY ). NOT supn≥1 suppnX

For a general problem instance.

Derived S-L expr. strictly larger than all current known inner bnds derived viaS-L coding techs. (Examples)

Derived S-L expr. subsume current known largest inner bounds, strictly enlargefor exmpls.

S1S2X2

YWY|X1X2

WY1Y2|X1X2

S1S2X2

YWY|X1X2

WY1Y2|X1X2

S1S2X2

YWY|X1X2

WY1Y2|X1X2

S1S2X2

YWY|X1X2

WY1Y2|X1X2

Multi-letter coding tech. : Why?? Necessary??

Prior work : Cover, El Gamal and Salehi (CES) [Nov. 1980] technique.Dueck’s example

[Nov. 1980](S1, S2) transmissible over MAC if

H(S1|S2) < I(X1;Y |X2, S2, U), H(S2|S1) < I(X2;Y |X1, S1, U)

H(S1, S2|K) < I(X1X2;Y |U), H(S1, S2) < I(X1X2;Y )

for a valid pmf WS1S2pUpX1|US1pX2|US2

WY |X1X2.

The Gacs - Korner part has to specially coded via a common conditionalcodebook.

Ignoring the GK part, treating it as soft correlation is strictly sub-optimal.

Conditional coding + separation : NO S-L long Markov chain (LMC)

S1(1) S1(2) S1(n)

X2(1) X2(2) X2(n)

U(1) U(2) U(n)

X1(1) X1(2) X1(n)

S2(1) S2(2) S2(n)

U(1) U(2) U(n)

BLOCK MAPPING

X1(i) = f1i(Sn1 ) and X2(i) = f2i(S

X1(i)− S1(i)− S2(i)−X2(i) does NOT hold.

In fact, X1 − US1 − US2 −X2 holds. SinceU(i) = gi(K

n), we potentially have

X1(i)− Sn1 − Sn

2 −X2(i),

permitting no more reduction.

“Gacs-Korner (GK) part permitting correlation viaan n−letter LMC, and n→∞.”

S1(1) S1(2) S1(n)

X2(1) X2(2) X2(n)

U(1) U(2) U(n)

X1(1) X1(2) X1(n)

S2(1) S2(2) S2(n)

U(1) U(2) U(n)

BLOCK MAPPING

X1(i)− Sn1 − Sn

2 −X2(i),

S1(1) S1(2) S1(n)

X2(1) X2(2) X2(n)

U(1) U(2) U(n)

X1(1) X1(2) X1(n)

S2(1) S2(2) S2(n)

U(1) U(2) U(n)

BLOCK MAPPING

X1(i)− Sn1 − Sn

2 −X2(i),

Stripped Gacs-Korner part ⇒ Constrained to S-L LMC

K X1 − US1 − US2 −X2 and hence

X1 − Sn1 − Sn

2 −X2 permitted.

P (K1 6= K2) = ξ > 0 is very tiny.

Constrained to

X1 − S1 − S2 −X2.

Discountnuous shrinkage of valid input PMFs.

Sub-optimality of Cover El Gamal Salehi rateregion.

P (K1 6= K2) = ξ > 0 is very tiny.

Constrained to

X1 − S1 − S2 −X2.

Discountnuous shrinkage of valid input PMFs.

Sub-optimality of Cover El Gamal Salehi rateregion.

Dueck’s ingenious example [March 1981]

S1S2X2

YWY|X1X2

Identifies a sequence of MACs and source pairs WY |X1X2and WS1S2 such that

P (S1 6= S2)→ 0 and, for all source-channel pairs, sufficiently far in thissequence

For any X1 − S1 − S2 −X2, we have I(X1X2;Y ) < H(S1S2).

Yet the source pair is transmissible over the MAC.

Import of Dueck’s example

Necessary to induce correlation via a block of symbols onto channel inputs.

This necessitates a multi-letter coding.

Focus of this talk

1. Desired coding structure.

2. Overall coding tech.

3. ??challenges.??

4. Tools.

Conditional coding of GK part

⇓Correlation induced via multi-letter LMC

Absence of GK part

⇓Constrained to S-L LMC

Induce l−letter correlation via approximate conditional coding.

Approximate conditional coding of near, but not perfect, GK parts

P (Kn1 6= Kn

2 ) = 1− (1− ξ)n → 1 as n→∞

FIX the BLOCK-LENGTH of (inner) conditional code to l.

Fixed B-L (inner) conditional code ⇒ non-vanishing Prob. of error

K1(1) K1(2) K1(l)

encoding

U1(1) U1(2) U1(l)

U l1 will be decoded with non-vanishing prob. of error.

Fixed B-L (inner) conditional code ⇒ non-vanishing Prob. of error

K1(1,1) K1(1,2) K1(1,l)

K1(m,1) K1(m,2) K1(m,l)

K1(t,1) K1(t,2) K1(t,l)

U1(1,1) U1(1,2) U1(1,l)

U1(m,1) U1(m,2) U1(m,l)

U1(t,1) U1(t,2) U1(t,l)

Row-by-row

encoding

Code over many many codewords of inner conditional code.

Number of rows m→∞.

No. of columns l remains constant.

Encoders employs identical inner codes and maps

K1(1,1) K1(1,2) K1(1,l)

K1(m,1) K1(m,2) K1(m,l)

K1(t,1) K1(t,2) K1(t,l)

U1(1,1) U1(1,2) U1(1,l)

U1(m,1) U1(m,2) U1(m,l)

U1(t,1) U1(t,2) U1(t,l)

Row-by-row

encoding

K2(1,1) K2(1,2) K2(1,l)

K2(m,1) K2(m,2) K2(m,l)

K2(t,1) K2(t,2) K2(t,l)

U2(1,1) U2(1,2) U2(1,l)

U2(m,1) U2(m,2) U2(m,l)

U2(t,1) U2(t,2) U2(t,l)

Row-by-row

encoding

Both encoders employ IDENTICAL inner codes and maps

Encoder 1

Encoder 2

Inner code error events

1. Cannot assign U−codeword to every l−length vector in K.

I τl =Prob. of unassigned U−codeword.

2. Encoders do NOT agree on a fraction of chosen U−codewords. How doyou handle this?

I Let ξ[l]:= P (Kl1 6= Kl

2) = 1− (1− ξ)l.

3. Decoder INCORRECTLY decodes a fraction of codewords on whichencoders agree on?

I Let gl denote prob of error of U−codebook on U − Y channel.

Inner code prob. of error : φ:= τl + ξ[l] + gl

How much more information needs to be sent?

Inner code decoded row by row. Let

G(1,1) G(1,2) G(1,l)

G(m,1) G(m,2) G(m,l)

G(t,1) G(t,2) G(t,l)

denote decoded version of K1,K2 matrices.

How much more information need to be sent?

Simple Fano-type upper bounding

Rows are IID. Treat as super-symbols.

G(1,1) G(1,2) G(1,l)

G(m,1) G(m,2) G(m,l)

G(t,1) G(t,2) G(t,l)

H(Sl1|Gl) ≤ H(Kl

1|Gl) +H(Sl1|Gl,Kl

≤ H(Kl1,1{Kl

1 6=Gl}|Gl) +H(Sl

1|Kl1)

≤ hb(φ) + lφ log |K|+ lH(S1|K1)

≤ l(LS(φ, |K|) +H(S1|K1))

LS(φ, |K| = Loss due to incorrect conditional coding

Simple Fano-type upper bounding

Rows are IID. Treat as super-symbols.

G(1,1) G(1,2) G(1,l)

G(m,1) G(m,2) G(m,l)

G(t,1) G(t,2) G(t,l)

H(Sl1|Gl) ≤ H(Kl

1|Gl) +H(Sl1|Gl,Kl

≤ H(Kl1,1{Kl

1 6=Gl}|Gl) +H(Sl

1|Kl1)

≤ hb(φ) + lφ log |K|+ lH(S1|K1)

≤ l(LS(φ, |K|) +H(S1|K1))

LS(φ, |K| = Loss due to incorrect conditional coding

Questions, Challenges

1. How do you communicate the rest of the information?

I Superposition coding.

2. How do you super-impose on codewords that induce an l−letter pmf?

I Achievable region should be in terms of pX1|U not pXl1|Ul .

Superposition coding over a matrix

U1(1,1) U1(1,2) U1(1,l)

U1(m,1) U1(m,2) U1(m,l)

U1(t,1) U1(t,2) U1(t,l)

U1(1,1) U1(1,2) U1(1,l)

U1(m,1) U1(m,2) U1(m,l)

U1(t,1) U1(t,2) U1(t,l)

Superposing on matrix requires optimizing over pXl1|Ul .

Desire S-L characterization. Desire suppX1|U.

Extract IID sub-vectors from this matrix and super-impose separate codebookson these sub-vectors.

Superposition coding over a matrix

U1(1,1) U1(1,2) U1(1,l)

U1(m,1) U1(m,2) U1(m,l)

U1(t,1) U1(t,2) U1(t,l)

U1(1,1) U1(1,2) U1(1,l)

U1(m,1) U1(m,2) U1(m,l)

U1(t,1) U1(t,2) U1(t,l)

Superposing on matrix requires optimizing over pXl1|Ul .

Desire S-L characterization. Desire suppX1|U.

Extract IID sub-vectors from this matrix and super-impose separate codebookson these sub-vectors.

The rows of Uj are independent and identically distributed.

U1(1,1) U1(1,2) U1(1,l)

U1(m,1) U1(m,2) U1(m,l)

U1(t,1) U1(t,2) U1(t,l)

U1(1,1) U1(1,2) U1(1,l)

U1(m,1) U1(m,2) U1(m,l)

U1(t,1) U1(t,2) U1(t,l)

Each row has been obtained by mapping a separate sub-block of sourcesymbols.

Interleaving [Shirani and Pradhan 2014]

The same argument holds for each sub-vectorsE(1, λ1(i)) · · ·E(m,λm(i)) : i = 1, · · · , l.

We therefore have l iid sub-vectors E(1, λ1(i)) · · ·E(m,λm(i)) : i = 1, · · · , l.

Randomly chosen co-ordinates are iid!!!

Co-ordinates with same color coded together within a block in the outer code.l such blocks.

Csiszar’s constant composition codes

Pick test channel pU to be type of sequences of length l.

The identical U−codebook chosen at both encoders is a constant compositioncode of type pU .

Interleaved vectors will be IID pU !!!!

Upperbound the difference using relationship between chosen and actualpmf

Let pUpX1|US1pX2|US2

be chosen pmf.

pUj= pU since symbols of CU chosen iid pU .

P (U1 6= U2) ≤ β

pXj |Uj Sj= pXj |USj

: By choice of coding technique. One can derive an upper

bound using these relations.

Upperbound the difference using relationship between chosen and actualpmf

Let pUpX1|US1pX2|US2

be chosen pmf.

pUj= pU since symbols of CU chosen iid pU .

P (U1 6= U2) ≤ β

pXj |Uj Sj= pXj |USj

: By choice of coding technique. One can derive an upper

bound using these relations.

Joint source channel coding using fixed-block length

Kj(t, 1 : l) is mapped to Uj(t, 1 : l) via fixed block-length code.

Induces a correlation between Kj and Uj

Unable to characterize this correlation analytically.

Propose to destroy this correlation by coding across two m× l matrices.

communicating correlated sources over user...

Documents