upper bounding the performance of arbitrary finite ldpc...

Upper Bounding the Performance ofArbitrary Finite LDPC Codes on

Binary Erasure ChannelsAn efficient exhaustion algorithm for

error-prone patternsC.-C. Wang, S.R. Kulkarni, and H.V. Poor

School of Electrical & Computer Engineering, Purdue Unversity

Department of Electrical Engineering, Princeton University

Wang, Kulkarni, & Poor – p. 1/24

ContentBrief introduction

Stopping sets in erasure channels. Hardness & Applications.

Existing approaches for upper / lower bounding thefixed

code performance:

A tree-basedupper bound forbit error rates.

Frame error rates and trapping sets.


Stopping Sets (SSs)Definition: a set of variable nodes such that the induced graph

contains no check node of degree 1.

Example: The binary (7,4) Hamming Code

i =i

1i

2i

3i

4i

5i

6i

7

1 2 3j =

��

��

��

��@

@@�

��@

@@PPPPPPP

HHHHH

@@@

(2, 5, 7) a validcodewordvs. (4, 6, 7) a purestopping set.

Hamming distance vs. Stopping distance





i =i

1yi

2i

3i

4yi

5i

6yi

7

1 2 3j =

��

��

��

��@

@@�

��@

@@PPPPPPP

HHHHH

@@@







i =i

1i

2i

3yi

4i

5yi

6yi

7

1 2 3j =

��

��

��

��@

@@�

��@

@@PPPPPPP

HHHHH

@@@




Upper Bounds vs. ExhaustingMinimum Stopping Sets

An n = 72 regular (3,6) LDPC code

10−3

10−2

10−1

100

10−20

10−15

10−10

10−5

100

erasure prob ε

ber

V60: Monte−CarloV60: UB


Upper Bounds vs. ExhaustingMinimum Stopping Sets

An n = 72 regular (3,6) LDPC code

10−3

10−2

10−1

100

10−20

10−15

10−10

10−5

100

erasure prob ε

ber

V60: Monte−CarloV60: UB

An NP-Complete Problem !!by Krishnanet al.


A Hard but Useful ProblemAs an upper bound: Guaranteed worst performance for extremely

low ber regimes.

As an SS exhaustion algorithm: Code annealing [Wang 06].

0.2 0.25 0.3 0.35 0.4 0.45 0.510

−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Erasure Probability ε

Err

or P

roba

bilit

y

Typical CLCL+d2optCL+CA(DA+CA)+CL+CA(DA+CA)+CL+CA, asym.Random Constr.Direct CA


Existing ApproachesEnsemble analysis:[Di et al. 02] avg. ber and FER curves,

[Amraoui et al. 04] the scaling law for water fall regions.

Fixed Code analysis:[Holzlöhneret al. 05] dual adaptive

importance sampling, [Stepanovet al. 06] instanton method

Enumerating bad patterns: [Richardson 03] error floors of

LDPC codes, [Yedidiaet al. 01] projection algebra, [Huet al.

04] the minimum distance of LDPC codes.

They are allinexhaustive enumeration.


The Bit-Oriented Detection

i =i

1i

2i

3i

4i

5i

6

1 2 3 4j =

��

@@@�

��@

@@PPPPPPP

��

@@@

��

@@@

Using

1: for erasure

0: for non-erasure



i =i

1i

2i

3i

4i

5i

6

1 2 3 4j =

��

@@@�

��@

@@PPPPPPP

��

@@@

��

@@@

Using

1: for erasure

0: for non-erasure

f2,0 = x2



i =i

1i

2i

3i

4i

5i

6

1 2 3 4j =

��

@@@�

��@

@@PPPPPPP

��

@@@

��

@@@

Using

1: for erasure

0: for non-erasure

f2,0 = x2

f2,1 = x2(x1 + x3)(x4 + x6)



i =i

1i

2i

3i

4i

5i

6

1 2 3 4j =

��

@@@�

��@

@@PPPPPPP

��

@@@

��

@@@

Using

1: for erasure

0: for non-erasure

f2,0 = x2

f2,1 = x2(x1 + x3)(x4 + x6)

f2,2 = x2 (x1( f5→4,1 + f6→4,1) + x3( f4→3,1 + f5→3,1))

· (x4( f3→3,1 + f5→3,1) + x6( f1→4,1 + f5→4,1))

= x2 (x1(x5 + x6) + x3(x4 + x5))

· (x4(x3 + x5) + x6(x1 + x5))



i =i

1i

2i

3i

4i

5i

6

1 2 3 4j =

��

@@@�

��@

@@PPPPPPP

��

@@@

��

@@@

Using

1: for erasure

0: for non-erasure

f2,0 = x2

f2,1 = x2(x1 + x3)(x4 + x6)

f2,2 = x2 (x1( f5→4,1 + f6→4,1) + x3( f4→3,1 + f5→3,1))

· (x4( f3→3,1 + f5→3,1) + x6( f1→4,1 + f5→4,1))

= x2 (x1(x5 + x6) + x3(x4 + x5))

· (x4(x3 + x5) + x6(x1 + x5))

f2 := liml→∞

f2,l = f2,2



i =i

1i

2i

3i

4i

5i

6

1 2 3 4j =

��

@@@�

��@

@@PPPPPPP

��

@@@

��

@@@

Using

1: for erasure

0: for non-erasure

f2,0 = x2

f2,1 = x2(x1 + x3)(x4 + x6)

f2,2 = x2 (x1( f5→4,1 + f6→4,1) + x3( f4→3,1 + f5→3,1))

· (x4( f3→3,1 + f5→3,1) + x6( f1→4,1 + f5→4,1))

= x2 (x1(x5 + x6) + x3(x4 + x5))

· (x4(x3 + x5) + x6(x1 + x5))

f2 := liml→∞

f2,l = f2,2

p2 = E{ f2}Wang, Kulkarni, & Poor – p. 7/24


i =i

1i

2i

3i

4i

5i

6

1 2 3 4j =

��

@@@�

��@

@@PPPPPPP

��

@@@

��

@@@

Using

1: for erasure

0: for non-erasure

Asymptotically, no repeated variables

f2,0 = x2

f2,1 = x2(x1 + x3)(x4 + x6)

f2,2 = x2 (x1( f5→4,1 + f6→4,1) + x3( f4→3,1 + f5→3,1))

· (x4( f3→3,1 + f5→3,1) + x6( f1→4,1 + f5→4,1))

= x2 (x1(x5 + x6) + x3(x4 + x5))

· (x4(x3 + x5) + x6(x1 + x5))

f2 := liml→∞

f2,l = f2,2

p2 = E{ f2}Wang, Kulkarni, & Poor – p. 7/24

Enumeration-Based LB & UBf2 = x2 (x1(x5 + x6) + x3(x4 + x5)) (x4(x3 + x5) + x6(x1 + x5))

= x1x2x6 + x2x3x4 + x1x2x4x5 + x2x3x5x6

Each product term corresponds to anirreducible stopping set.



= x1x2x6 + x2x3x4 + x1x2x4x5 + x2x3x5x6


LB: Graph-based search [Richardson 03], iteration-based

relaxation [Yedidiaet al. 01]

f2,LBa = x2x3x4

LBa = E{ f2,LB} = ǫ3

f2,LBb= x1x2x4x5

LBb = E{ f2,LBb} = ǫ4



= x1x2x6 + x2x3x4 + x1x2x4x5 + x2x3x5x6


LB: Graph-based search [Richardson 03], iteration-based

relaxation [Yedidiaet al. 01]

f2,LBa = x2x3x4

LBa = E{ f2,LB} = ǫ3

f2,LBb= x1x2x4x5

LBb = E{ f2,LBb} = ǫ4

UB: Iteration-based relaxation [Yedidiaet al. 01]

f2,2 = x2x3x4 + x1x2x4x5 + x1x2x6 + x2x3x5x6

≤ x21x4 + 1x2x41 + 1x2x6 + x21 · 1x6 = x2x4 + x2x6

UB = 2ǫ2 − ǫ3Wang, Kulkarni, & Poor – p. 8/24

An Upper Bound

- i2 - i3 - i2 - i4 - i4�� @@ �� @@

��HHHH

- i1 - i3!!!!

aaaa

fa

- i3 - i4

�� @@

- i2 i - i4�� @@

- i1 - i3�

��Q

QQ

fb


An Upper Bound

- i2 - i3 - i2 - i4 - i4�� @@ �� @@

��HHHH

- i1 - i3!!!!

aaaa

fa

- i3 - i4

�� @@

- i2 i - i4�� @@

- i1 - i3�

��Q

QQ

fb

fa = fb

EDE{ fa} < pe = E{ fa} = E{ fb} ≤ EDE{ fb}


An Upper Bound

- yi2 - i3 - yi2 - i4 - i4�� @@ �� @@

��HHHH

- yi1 - i3!!!!

aaaa

fa

- i3 - i4

�� @@

- i2 i - i4�� @@

- yi1 - i3�

��Q

QQ

fb

fa = fb



An Upper Bound

- yi2 - i3 - yi2 - i4 - i4�� @@ �� @@

��HHHH

- yi1 - i3!!!!

aaaa

fa

y- i3 - i4

�� @@

- i2 i - i4�� @@

- yi1 y- i3�

��Q

QQ

fb

fa = fb



An Upper Bound (Cont’d)

Theorem 1 Suppose all messages entering any variable node are

independent, or equivalently, theyoungest common ancestorsof all

pairs of repeated bits arecheck nodes.

EDE{ f } ≥ E{ f }

Theorem 2 This upper bound is tight in order.

EDE{ f }(ǫ) = O(E{ f }(ǫ)), ∀ǫ


A Pivoting Rule

f = x0 · f |x0=1 + f |x0=0


A Pivoting Rule

f = x0 · f |x0=1 + f |x0=0

lf

��

JJ

JJQQ

JJ

x0

-x0 lf |x0=1

��

JJ

JJHH

JJ

1

lf |x0=0

��

JJ

JJHH

JJ

0

f

!!!!aaaa

(a) Original (b) Decoupled


The Two-Stage Algorithm

6j-

6

��*j- HHHY

j-

6 6

��j- @@Ij- ��j- @@Ij-

f∞



f∞



f∞ ≤ fFINITE



f∞ ≤ fFINITE = f̃FINITE




≤w. the same order

≤pe = E{ f∞} E{ fFINITE} EDE{ f̃FINITE}




≤w. the same order

≤pe = E{ f∞} E{ fFINITE} EDE{ f̃FINITE}

yi yi yi i yi yi

�� @

@@@

@@@

@@PPPPPPP

��

��@

@@

��96

@@@I


A Narrowing SearchTheorem 3 (Asymptotically Tight) With an“optimal" growing

module, we have

UB(ǫ) = O(pe(ǫ)).


A Narrowing SearchTheorem 3 (Asymptotically Tight) With an“optimal" growing

module, we have

UB(ǫ) = O(pe(ǫ)).

Theorem 4 (Exhaustive Enumeration)

d := min{size(x)|∀x such thatf̃FINITE(x) = 1}. Then the

stopping distance≥ d.

If there exists such a minimumx being a SS, then ALL minimum

SSs are in the set of all minimumx.

The exhaustive list of minimum SSs leads to a lower boundtight

in bothorderandmultiplicity.




≤ ≤pe = E{ f∞} E{ fFINITE} EDE{ f̃FINITE}

yi yi yi i yi yi

�� @

@@@

@@@

@@PPPPPPP

��

��@

@@

��96

@@@I


Numerical ExperimentsThe (7,4,3) Hamming code:

H =

0 0 1 1 1 0 1

0 1 0 1 0 1 1

1 0 0 0 1 1 1



H =

0 0 1 1 1 0 1

0 1 0 1 0 1 1

1 0 0 0 1 1 1

∀i ∈ [1, 7], ∀ǫ ∈ (0, 1], UBi(ǫ)pi(ǫ)

≤ 1.3.



H =

0 0 1 1 1 0 1

0 1 0 1 0 1 1

1 0 0 0 1 1 1

∀i ∈ [1, 7], ∀ǫ ∈ (0, 1], UBi(ǫ)pi(ǫ)

≤ 1.3.

The (23,12,7) Golay code:

H = [H′I] in which H

′ =

1 0 0 1 1 1 0 0 0 1 1 1

1 0 1 0 1 1 0 1 1 0 0 1

1 0 1 1 0 1 1 0 1 0 1 0

1 0 1 1 1 0 1 1 0 1 0 0

1 1 0 0 1 1 1 0 1 1 0 0

1 1 0 1 0 1 1 1 0 0 0 1

1 1 0 1 1 0 0 1 1 0 1 0

1 1 1 0 0 1 0 1 0 1 1 0

1 1 1 0 1 0 1 0 0 0 1 1

1 1 1 1 0 0 0 0 1 1 0 1

0 1 1 1 1 1 1 1 1 1 1 1


Numerical Experiments (cont’d)The (23,12,7) Golay code:

bit 0, (4*,75*)

10−3

10−2

10−1

100

10−10

10−8

10−6

10−4

10−2

100

erasure prob ε

ber

V0: MC−SV0: UBV0: C−UBV0: LB


Numerical Experiments (cont’d)A fixed, finite LDPC code with var deg. 3, chk deg. 6,n = 50:

bit 0, (4*,1*)

10−3

10−2

10−1

100

10−15

10−10

10−5

100

erasure prob ε

ber




bit 26, (6*,2*)

10−3

10−2

10−1

100

10−15

10−10

10−5

100

erasure prob ε

ber




bit 19, (7*,10 → 5*)

10−3

10−2

10−1

100

10−15

10−10

10−5

100

erasure prob ε

ber



Frame Error Rates and IrregularCodes

Frame error rate (FER):fFER = ∑ni=1 fi.




Efficiency:

More cycles=⇒ less efficiency. (The Golay code is the

worst.)

The largern, the easier for our algorithm.




Efficiency:

More cycles=⇒ less efficiency. (The Golay code is the

worst.)

The largern, the easier for our algorithm.

Experimental results

Consider codes ofn = 500–1000.

For regular codes,dSS ≤ 12 can be exhausted.

((51212 ) = 5.9 × 1023 trials)

TheFERof irregular codessuits the algorithm most.

dSS ≤ 13 can be exhausted. ((51213 ) = 2.5 × 1025 trials)


The FER of A Rate 1/2 IrregularCode

λ(x) = 0.416667x + 0.166667x2 + 0.416667x5

ρ(x) = x5. 61% variable nodes of degree 2

A rate 1/2n = 572 irregular LDPC code, (order, multi)=(13*,104*)

0.2 0.25 0.3 0.35 0.4 0.45 0.510

−7

10−6

10−5

10−4

10−3

10−2

10−1

100

Erasure Probability ε

Err

or P

roba

bilit

y

Typical CLCL+d2optCL+CA(DA+CA)+CL+CA(DA+CA)+CL+CA, asym.Random Constr.Direct CA


SummaryUpper bounding and exhausting the bad patterns

An NP-completeproblem butfeasiblefor at least short practical

LDPC codes.




LDPC codes.

Taking advantages of thetree-likestructure.




LDPC codes.


Suitable for bothFERandber, punctured codes, unequal

sub-channel analysis, non-sparse codes, etc.




LDPC codes.




Our algorithm can be easily modified fortrapping sets.




LDPC codes.





A useful tool forfinite code optimization.




LDPC codes.





A useful tool forfinite code optimization.

A starting point for more efficient algorithms.


Thank you for the attention.


Searching for Trapping SetsWe define thek-out trapping set, namely, the induced graph hask

check node of degree 1.

yi i i i yi yi

�� @

@@@

@@@

@@PPPPPPP

��

��@

@@

��9 @@@

Theorem 5 Consider a fixedk. Determining thek-out trapping

distance is NP-complete.

Our algorithm can be modified for trapping set exhaustion.


Some NotesComplexity of evaluatingUB: O(|T | log(|T |)), even for

arbitrarily smallǫ.




A pleasant byproduct: anexhaustivelist of minimum SS leading

to a lower bound tight in bothorderandmultiplicity.






Complexity and performance tradeoff.







Puncturedvs. ShortenedCodes







Puncturedvs. ShortenedCodes

A partitioned approach— a hybrid search

E{ f } = ∑j

E{Aj}E{ f |Aj}.

Ex: A1 = {x3 = 0}, A2 = {x3 = 1, x7 = 0}, and

A3 = {x3 = 1, x7 = 1}.


The Tree Converting Algorithm1: repeat2: Find the next leaf variable node, sayxj.

3: if there exists another non-leafxj in T then4: if theyoungest common ancestorof the leafxj and existing

non-leafxj, denoted asyca(xj), is a check nodethen5: Include the newxj in T .

6: else if yca(xj) is avariable nodethen7: Do thepivotingconstruction.

8: end if9: end if

10: Construct the immediate children ofxj.

11: until the size ofT exceeds the preset limit.


upper bounding the performance of arbitrary finite ldpc...

Documents