daniel j. costello, jr. - ece.engin.umich.edu · daniel j. costello, jr. dept. of electrical...

146
Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University of Notre Dame University of Michigan, Nov. 17, 2016 Research Collaborators: David Mitchell, Michael Lentmaier, and Ali Pusane

Upload: others

Post on 23-Aug-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

Daniel J. Costello, Jr.

Dept. of Electrical Engineering,

Spatially Coupled LDPC Codes:Is This What Shannon Had In Mind?

University of Notre Dame

University of Michigan, Nov. 17, 2016

Research Collaborators: David Mitchell,Michael Lentmaier, and Ali Pusane

Page 2: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Outline

LDPC Block Codes

Spatially Coupled LDPC Codes

Parity-check matrix and Tanner graph representations, minimumdistance bounds, iterative decoding thresholds, protograph-based constructions

Protograph representation, edge-spreading construction, termination

Iterative decoding thresholds, threshold saturation, minimum distance

Practical Considerations

1

Window decoding, performance, latency, and complexity comparisonsto LDPC block codes, rate-compatibility, implementation aspects

From Shannon to Modern Coding Theory

Channel capacity, structured codes, random codes, LDPC codes

Page 3: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Claude Elwood ShannonApr. 30, 1916 – Feb. 24, 2001Father of Information Theory

Shannon's Legacy

2

Page 4: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Shannon's Legacy

3

Page 5: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Three Great Successes of Information Theory

• Source Coding for Data Compression

• Secret Coding (Cryptography) for Data Security

• Channel Coding for Data Reliability (the focus of thispresentation)

Shannon's Legacy

4

Page 6: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

1948

Shannon's Legacy

5

Page 7: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

1948

Shannon's Legacy

Power efficiency, or signal-to-noise ratio (SNR), represents the

quality of the channel and isdefined as the ratio of the

average information symbolenergy Eb to the single-sided

power spectral density No of thenoise It is usually given in decibels (dB).

5

Page 8: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

1948

Shannon's Legacy

Bandwidth efficiency is normalized to thenumber of information bits transmitted

per 2-dimensional signal (bits/2D)

For example, uncoded 8PSK transmitsbits/2D and uncoded 16QAM transmits

bits/2D

5

Page 9: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

1948

Shannon's Legacy

Shannon’s capacity theorem lowerbounds the channel quality for whichthere exists a code that can achieve

arbitrarily reliable transmission. For theAWGN channel, this lower bound is

5

Page 10: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

1948

Coding theory playing fieldCoding theory playing field

Shannon's Legacy

6

Page 11: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Shannon's Legacy

7

Page 12: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Shannon showed that random codes with large blocklength can achieve capacity, but...

The coding dilemma

8

Page 13: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Shannon showed that random codes with large blocklength can achieve capacity, but...

… code structure (algebraic/topological) is required inorder to permit decoding with reasonable complexity.

The coding dilemma

8

Page 14: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

“Almost all codes are good...

Shannon showed that random codes with large blocklength can achieve capacity, but...

… code structure (algebraic/topological) is required inorder to permit decoding with reasonable complexity.

The coding dilemma

8

Page 15: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

“Almost all codes are good... except those we can think of.”

Shannon showed that random codes with large blocklength can achieve capacity, but...

… code structure (algebraic/topological) is required inorder to permit decoding with reasonable complexity.

The coding dilemma

8

Page 16: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

“Almost all codes are good... except those we can think of.”

Solution: Construct random-like codes with just enough structure to allow efficient decoding

Shannon showed that random codes with large blocklength can achieve capacity, but...

… code structure (algebraic/topological) is required inorder to permit decoding with reasonable complexity.

Modern Coding Theory

The coding dilemma

8

Page 17: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Cut

off R

ate

LDPC Codes: motivation (for a target BER 10-5)

9

Page 18: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Uncoded BPSK/QPSK

Cut

off R

ate

LDPC Codes: motivation (for a target BER 10-5)

9

Page 19: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Hamming (7,4)

Uncoded BPSK/QPSK

Cut

off R

ate

LDPC Codes: motivation (for a target BER 10-5)

9

Page 20: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Golay (24,12)

Hamming (7,4)

Uncoded BPSK/QPSK

Cut

off R

ate

LDPC Codes: motivation (for a target BER 10-5)

9

Page 21: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Golay (24,12)

BCH (255,123)

Hamming (7,4)

Uncoded BPSK/QPSK

Cut

off R

ate

LDPC Codes: motivation (for a target BER 10-5)

9

Page 22: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Golay (24,12)

BCH (255,123)

RS (64,32)

Hamming (7,4)

Uncoded BPSK/QPSK

Cut

off R

ate

LDPC Codes: motivation (for a target BER 10-5)

9

Page 23: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Golay (24,12)

BCH (255,123)

RS (64,32)

Viterbi(2,1,6)

Hamming (7,4)

Uncoded BPSK/QPSK

Cut

off R

ate

LDPC Codes: motivation (for a target BER 10-5)

9

Page 24: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Golay (24,12)

BCH (255,123)

RS (64,32)

Pioneer QLI (2,1,31) Viterbi

(2,1,6)Hamming

(7,4)

Uncoded BPSK/QPSK

Cut

off R

ate

LDPC Codes: motivation (for a target BER 10-5)

9

Page 25: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2

1

00 2 4 6 8 10 12-2.0

Cap

acity

Boun

d

BPSK/QPSK capacity bounds

Golay (24,12)

BCH (255,123)

RS (64,32)

LDPC (107) IR

Pioneer QLI (2,1,31) Viterbi

(2,1,6)

Turbo (65536,18)

Hamming (7,4)

Uncoded BPSK/QPSK

Cut

off R

ate

LDPC (8096,3,6)

LDPC Codes: motivation (for a target BER 10-5)

9

Page 26: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Turbo codes use a long pseudorandom interleaver

3G and 4G telephonystandards HSPA, EV-DO,LTE, satellite DVB-RCS,Mars ReconnaissanceRover, WiMAX, and so on.

Random-like codes (2000s - today)

10

Page 27: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

DVB-S2, ITU-T G.hnstandard (data networkingover power lines, phone lines,and coaxial cables),10GBase-T Ethernet, Wi-Fistandards 802.11, and so on.

Turbo codes use a long pseudorandom interleaver

3G and 4G telephonystandards HSPA, EV-DO,LTE, satellite DVB-RCS,Mars ReconnaissanceRover, WiMAX, and so on.

Low-density parity-check (LDPC) codes are defined on a large sparsegraph

Random-like codes (2000s - today)

10

Page 28: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Outline

LDPC Block Codes

Spatially Coupled LDPC Codes

Parity-check matrix and Tanner graph representations, minimumdistance bounds, iterative decoding thresholds, protograph-based constructions

Protograph representation, edge-spreading construction, termination

Iterative decoding thresholds, threshold saturation, minimum distance

Practical Considerations

Window decoding, performance, latency, and complexity comparisonsto LDPC block codes, rate-compatibility, implementation aspects

From Shannon to Modern Coding Theory

Channel capacity, structured codes, random codes, LDPCcodes

11

Page 29: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

LDPC Block Codes

Definition by parity-check matrix: [Gallager, '62]

Code:(J,K)-regular LDPC block code:

Bipartite graph representation: [Tanner, '81]

n = 20 variable nodes of degree J = 3

l = 15 check nodes of degree K = 4

12

Page 30: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

LDPC Block Codes

Definition by parity-check matrix: [Gallager, '62]

Code:(J,K)-regular LDPC block code:

Bipartite graph representation: [Tanner, '81]

n = 20 variable nodes of degree J = 3

l = 15 check nodes of degree K = 4

Graph-based codes can be decoded iteratively with low complexity byexchanging messages in the graph using Belief Propagation (BP).

12

Page 31: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

For an asymptotically good code ensemble, the minimum distance grows linearly with the block length n

Minimum Distance Growth Rates of (J,K)-Regular LDPC Block Code Ensembles

13

Page 32: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

where is calledthe typical minimumdistance ratio, orminimum distancegrowth rate.

For an asymptotically good code ensemble, the minimum distance grows linearly with the block length n

Minimum Distance Growth Rates of (J,K)-Regular LDPC Block Code Ensembles

(J,K)-regular block codeensembles areasymptotically good, i.e.,

13

Page 33: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

As the density of (J,K)-regular ensembles increases,approaches theGilbert-Varshamovbound.

where is calledthe typical minimumdistance ratio, orminimum distancegrowth rate.

For an asymptotically good code ensemble, the minimum distance grows linearly with the block length n

Minimum Distance Growth Rates of (J,K)-Regular LDPC Block Code Ensembles

(J,K)-regular block codeensembles areasymptotically good, i.e.,

13

Page 34: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

[RU01] T. J. Richardson, and R. Urbanke, “The capacity of low-density parity-check codes under messagepassing decoding”, IEEE Transactions on Information Theory, vol. 47 no. 2, Feb. 2001.

AWGNC thresholdsBEC thresholds

Iterative decoding thresholds can be calculated for (J,K)-regularLDPC block code ensembles using density evolution (DE).

Thresholds of (J,K)-regular LDPC Block Code Ensembles

14

Page 35: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

[RU01] T. J. Richardson, and R. Urbanke, “The capacity of low-density parity-check codes under messagepassing decoding”, IEEE Transactions on Information Theory, vol. 47 no. 2, Feb. 2001.

There exists a relatively large gap to capacity.

AWGNC thresholdsBEC thresholds

Iterative decoding thresholds can be calculated for (J,K)-regularLDPC block code ensembles using density evolution (DE).

Thresholds of (J,K)-regular LDPC Block Code Ensembles

14

Page 36: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

[RU01] T. J. Richardson, and R. Urbanke, “The capacity of low-density parity-check codes under messagepassing decoding”, IEEE Transactions on Information Theory, vol. 47 no. 2, Feb. 2001.

There exists a relatively large gap to capacity.

AWGNC thresholdsBEC thresholds

Iterative decoding thresholds can be calculated for (J,K)-regularLDPC block code ensembles using density evolution (DE).

Thresholds of (J,K)-regular LDPC Block Code Ensembles

Iterative decoding thresholds get further from capacity as the graphdensity increases.

14

Page 37: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Large LDPC codes can be obtained from a small base parity-checkmatrix B by replacing each nonzero entry in B with an M x M permutation matrix, where M is the lifting factor.

Protographs (Matrix Description)

15

Page 38: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Large LDPC codes can be obtained from a small base parity-checkmatrix B by replacing each nonzero entry in B with an M x M permutation matrix, where M is the lifting factor.

length 6M = 24rate R = 1/2

Example: Irregular code with M = 4

Protographs (Matrix Description)

15

Page 39: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Large LDPC codes can be obtained from a small base parity-checkmatrix B by replacing each nonzero entry in B with an M x M permutation matrix, where M is the lifting factor.

length 6M = 24rate R = 1/2

Example: Irregular code with M = 4

Irregularcodes havevariable rowand columnweights(check nodeand variablenode degrees)

Protographs (Matrix Description)

15

Page 40: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Protographs are often represented using a bipartite Tanner graph

Protographs (Graphical Description)

3 check nodes

6 variable nodes

[Tho05] J. Thorpe, “Low-Density Parity-Check (LDPC) codes constructed fromprotographs”, Jet Propulsion Laboratory INP Progress Report, Vol. 42-154 Aug. 2003.

16

Page 41: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Protographs are often represented using a bipartite Tanner graph

Protographs (Graphical Description)

3 check nodes

6 variable nodes

The collection of all possible parity-check matrices with lifting factor M forms a code ensemble, where all the codes share a common structure

[Tho05] J. Thorpe, “Low-Density Parity-Check (LDPC) codes constructed fromprotographs”, Jet Propulsion Laboratory INP Progress Report, Vol. 42-154 Aug. 2003.

16

Page 42: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

.Quasi-cyclic (QC) LDPC codes are of great interest in practice, sincethey have efficient encoder and decoder implementations

Quasi-Cyclic LDPC Codes

17

Page 43: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

.Quasi-cyclic (QC) LDPC codes are of great interest in practice, sincethey have efficient encoder and decoder implementations

Example: protograph construction of a (2,3)-regular QC-LDPC block code

Quasi-Cyclic LDPC Codes

17

Page 44: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

.Quasi-cyclic (QC) LDPC codes are of great interest in practice, sincethey have efficient encoder and decoder implementations

Example: protograph construction of a (2,3)-regular QC-LDPC block code

For QC codes, the permutationmatrices are shifted identities

Quasi-Cyclic LDPC Codes

17

Page 45: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

[DDJA09] D. Divsalar, S. Dolinar, C. Jones, and K. Andrews, “Capacity-approaching protographcodes”, IEEE Journal on Select Areas in Communications, vol. 27, no. 6 Aug. 2009.

Multi-Edge Protographs

Protographs can have repeated edges (corresponding to integer valuesgreater than one in B)

Note that this makes nosense without lifting

18

Page 46: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

[DDJA09] D. Divsalar, S. Dolinar, C. Jones, and K. Andrews, “Capacity-approaching protographcodes”, IEEE Journal on Select Areas in Communications, vol. 27, no. 6 Aug. 2009.

Multi-Edge Protographs

Protographs can have repeated edges (corresponding to integer valuesgreater than one in B)

Repeated edges in aprotograph correspond tousing sums of permutationmatrices to form LDPC codeensembles

Note that this makes nosense without lifting

18

Page 47: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

[DDJA09] D. Divsalar, S. Dolinar, C. Jones, and K. Andrews, “Capacity-approaching protographcodes”, IEEE Journal on Select Areas in Communications, vol. 27, no. 6 Aug. 2009.

Multi-Edge Protographs

Protographs can have repeated edges (corresponding to integer valuesgreater than one in B)

Repeated edges in aprotograph correspond tousing sums of permutationmatrices to form LDPC codeensembles

denser graphs!can also be QC (using circulant matrices)!

Note that this makes nosense without lifting

18

Page 48: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

2e variablenodes

Rate Threshold Capacity Distancegrowth

rate

1/2 0.628 0.187 0.01450

2/3 1.450 1.059 0.00582

3/4 2.005 1.628 0.00323

4/5 2.413 2.040 0.00207

5/6 2.733 2.362 0.00145

6/7 2.993 2.625 0.00108

[DDJA09] D. Divsalar, S. Dolinar, C. Jones, and K. Andrews, “Capacity-approaching protographcodes”, IEEE Journal on Select Areas in Communications, vol. 27, no. 6 Aug. 2009.

'Good' Protographs

Ensemble average properties can be easily calculated from a protograph,thus simplifying the construction of 'good' code ensembles.

Iterative decoding thresholds close to capacity for irregular protographsMinimum distance growing linearly with block length (asymptotically good) for regular and some irregular protographs

19

Page 49: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

irregular threshold

regular threshold

RegularLDPC

Irregular LDPC

Regular vs. Irregular LDPC codes

20

Page 50: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

thresholds far from capacity

“Regular” LDPC codes:

low error floors

irregular threshold

regular threshold

XRegularLDPC

Irregular LDPC

Regular vs. Irregular LDPC codes

structure aids implementation

20

Page 51: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

thresholds far from capacity

“Regular” LDPC codes:

low error floors

not suitable for severely powerconstrained applications

irregular threshold

regular threshold

XRegularLDPC

Irregular LDPC

Regular vs. Irregular LDPC codes

structure aids implementation

20

Page 52: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

thresholds far from capacity

“Regular” LDPC codes:

low error floors

not suitable for severely powerconstrained applications

thresholds close to capacity

“Irregular” LDPC codes:

visible error floors

irregular threshold

regular threshold

X

X

RegularLDPC

Irregular LDPC

Regular vs. Irregular LDPC codes

structure aids implementation

Less desirable structure X

20

Page 53: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

thresholds far from capacity

“Regular” LDPC codes:

low error floors

not suitable for severely powerconstrained applications

thresholds close to capacity

“Irregular” LDPC codes:

visible error floors

not suitable for applications thatrequire very low error rates

irregular threshold

regular threshold

X

X

RegularLDPC

Irregular LDPC

Regular vs. Irregular LDPC codes

structure aids implementation

Less desirable structure X

20

Page 54: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

thresholds far from capacity

“Regular” LDPC codes:

low error floors

not suitable for severely powerconstrained applications

thresholds close to capacity

“Irregular” LDPC codes:

visible error floors

not suitable for applications thatrequire very low error rates

Spatially coupled LDPC codescombine all of the positive features!

irregular threshold

regular threshold

X

X

RegularLDPC

Irregular LDPC

Regular vs. Irregular LDPC codes

structure aids implementation

Less desirable structure X

20

Page 55: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Outline

Spatially Coupled LDPC Codes

Protograph representation, edge-spreading construction, termination

Iterative decoding thresholds, threshold saturation, minimum distance

LDPC Block Codes

Practical Considerations

21

Parity-check matrix and Tanner graph representations, minimumdistance bounds, iterative decoding thresholds, protograph-based constructions

Window decoding; performance, latency, and complexity comparisonsto LDPC block codes; rate-compatibility; implementation aspects

Page 56: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Spatially Coupled Protographs

...

Consider transmission of consecutive blocks (protograph representation):

... (3,6)-regularLDPC-BC

base matrix

22

Page 57: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Spatially Coupled Protographs

...

Consider transmission of consecutive blocks (protograph representation):

... (3,6)-regularLDPC-BC

base matrix

Blocks are spatially coupled (introducing memory) by spreading edges over time:

22

Page 58: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Spatially Coupled Protographs

...

Consider transmission of consecutive blocks (protograph representation):

... (3,6)-regularLDPC-BC

base matrix

Blocks are spatially coupled (introducing memory) by spreading edges over time:

Spreading constraint: ( has size )

22

Page 59: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Transmission of consecutive spatially coupled (SC) blocks results in aconvolutional protograph:

Spatially Coupled Protographs

23

Page 60: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Transmission of consecutive spatially coupled (SC) blocks results in aconvolutional protograph:

Spatially Coupled Protographs

23

Page 61: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Transmission of consecutive spatially coupled (SC) blocks results in aconvolutional protograph:

Spatially Coupled Protographs

23

Page 62: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Transmission of consecutive spatially coupled (SC) blocks results in aconvolutional protograph:

... ...

Spatially Coupled Protographs

23

Page 63: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Transmission of consecutive spatially coupled (SC) blocks results in aconvolutional protograph:

... ...

The bi-infinite convolutional protograph corresponds to a bi-infiniteconvolutional base matrix:

Spatially Coupled Protographs

Constraint length:

Rate:

23

Page 64: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

SC-LDPC Code Ensembles

An ensemble of (3,6)-regular SC-LDPC codes can be created from theconvolutional protograph by the graph lifting operation

24

Page 65: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

SC-LDPC Code Ensembles

An ensemble of (3,6)-regular SC-LDPC codes can be created from theconvolutional protograph by the graph lifting operation

24

Page 66: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

SC-LDPC Code Ensembles

An ensemble of (3,6)-regular SC-LDPC codes can be created from theconvolutional protograph by the graph lifting operation

Graph lifting: is an permutation matrix

24

Page 67: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

SC-LDPC Code Ensembles

An ensemble of (3,6)-regular SC-LDPC codes can be created from theconvolutional protograph by the graph lifting operation

Graph lifting: is an permutation matrix

If each permutation matrix is circulant, the codes are quasi-cyclic

24

Page 68: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Code rate:

Consider terminating to a (block code) base matrix of length Lbv:

Terminated Spatially Coupled Codes

25

Page 69: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Code rate:

For large L, RL approaches the unterminated code rate .

Consider terminating to a (block code) base matrix of length Lbv:

Terminated Spatially Coupled Codes

25

Page 70: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Example: (3,6)-regular base matrix , ms = 2, L = 4, R4 = 1/4

Code rate:

For large L, RL approaches the unterminated code rate .

Consider terminating to a (block code) base matrix of length Lbv:

(check node degrees lowerat the ends)

Terminated Spatially Coupled Codes

25

Page 71: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Example: (3,6)-regular base matrix , ms = 2, L = 4, R4 = 1/4

Code rate:

For large L, RL approaches the unterminated code rate .

Consider terminating to a (block code) base matrix of length Lbv:

(check node degrees lowerat the ends)

Terminated Spatially Coupled Codes

Codes can be lifted to different lengths and rates by varying M and L .

25

Page 72: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

...

26

Thresholds of SC-LDPC Codes

Page 73: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

10 20 30 40 50 60 70 80 90 10010

-6

10-4

10-2

100

10 iterations

t

p

Evolution of message probabilities (L = 100):

26

Thresholds of SC-LDPC Codes

10 20 30 40 50 60 70 80 90 100

p

t

Page 74: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

Evolution of message probabilities (L = 100):

10 20 30 40 50 60 70 80 90 10010

-6

10-4

10-2

100

20 iterations

t

p

26

Thresholds of SC-LDPC Codes

10 20 30 40 50 60 70 80 90 100

p

t

Page 75: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

Evolution of message probabilities (L = 100):

10 20 30 40 50 60 70 80 90 10010

-6

10-4

10-2

100

50 iterations

t

p

26

Thresholds of SC-LDPC Codes

10 20 30 40 50 60 70 80 90 100

p

t

Page 76: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

Evolution of message probabilities (L = 100):

10 20 30 40 50 60 70 80 90 10010

-6

10-4

10-2

100

100 iterations

t

p

26

Thresholds of SC-LDPC Codes

10 20 30 40 50 60 70 80 90 100

p

t

Page 77: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

Evolution of message probabilities (L = 100):

10 20 30 40 50 60 70 80 90 10010

-6

10-4

10-2

100

200 iterations

t

p

26

Thresholds of SC-LDPC Codes

10 20 30 40 50 60 70 80 90 100

p

t

Page 78: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

Evolution of message probabilities (L = 100):

10 20 30 40 50 60 70 80 90 10010

-6

10-4

10-2

100

300 iterations

t

p

26

Thresholds of SC-LDPC Codes

10 20 30 40 50 60 70 80 90 100

p

t

300 iterations

Page 79: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

Evolution of message probabilities (L = 100):

10 20 30 40 50 60 70 80 90 10010

-6

10-4

10-2

100

340 iterations

t

p

26

Thresholds of SC-LDPC Codes

10 20 30 40 50 60 70 80 90 100

p

t

340 iterations

“wave-like decoding”

Page 80: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Variable nodes all have the same degree as the block code.

Check nodes with lower degrees (at the ends) improve the BP decoder.

Evolution of message probabilities (L = 100):

10 20 30 40 50 60 70 80 90 10010

-6

10-4

10-2

100

340 iterations

t

p

Note: the fraction of lower degree nodes tends to zero asi.e., the codes are asymptotically regular.

26

Thresholds of SC-LDPC Codes

10 20 30 40 50 60 70 80 90 100

p

t

340 iterations

“wave-like decoding”

Page 81: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Density evolution can be applied to the protograph-based ensembles with [Sridharan et al. '04]:

Example: BEC

27

Thresholds of SC-LDPC Codes

Page 82: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

L=4, gap=0.115

Density evolution can be applied to the protograph-based ensembles with [Sridharan et al. '04]:

Example: BEC

27

Thresholds of SC-LDPC Codes

Page 83: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

L=4, gap=0.115

Density evolution can be applied to the protograph-based ensembles with [Sridharan et al. '04]:

Example: BEC

L=10, gap=0.095

27

Thresholds of SC-LDPC Codes

Page 84: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

L=4, gap=0.115

Density evolution can be applied to the protograph-based ensembles with [Sridharan et al. '04]:

Example: BEC

L=10, gap=0.095

(3,6)-regular block code:

27

Thresholds of SC-LDPC Codes

Page 85: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

BEC AWGN

Iterative decoding thresholds (protograph-based ensembles)

We observe a significant improvement in the thresholds of SC-LDPC codes compared to the associated LDPC block codes (LDPC-BCs) due to the lowerdegree check nodes at the ends of the graph and the wave-like decoding.

28

[LSCZ10] M. Lentmaier, A. Sridharan, D. J. Costello, Jr., and K.Sh. Zigangirov, “Iterative decoding thresholdanalysis for LDPC convolutional codes,” IEEE Trans. Inf. Theory, 56:10, Oct. 2010.

SC

Thresholds of SC-LDPC Codes

Page 86: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

BEC AWGN

Iterative decoding thresholds (protograph-based ensembles)

We observe a significant improvement in the thresholds of SC-LDPC codes compared to the associated LDPC block codes (LDPC-BCs) due to the lowerdegree check nodes at the ends of the graph and the wave-like decoding.

In contrast to LDPC-BCs, the iterative decoding thresholds of SC-LDPC codes improve as the graph density increases.

28

[LSCZ10] M. Lentmaier, A. Sridharan, D. J. Costello, Jr., and K.Sh. Zigangirov, “Iterative decoding thresholdanalysis for LDPC convolutional codes,” IEEE Trans. Inf. Theory, 56:10, Oct. 2010.

SC

Thresholds of SC-LDPC Codes

Page 87: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are SC-LDPC Codes Better?

...

29

Page 88: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are SC-LDPC Codes Better?

...

29

Page 89: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are SC-LDPC Codes Better?

...

29

Page 90: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are SC-LDPC Codes Better?

...

29

Page 91: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are SC-LDPC Codes Better?

...

29

Page 92: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are SC-LDPC Codes Better?

...

The threshold saturates (converges) to a fixed value numericallyindistinguishable from the maximum a posteriori (MAP) threshold of the (J, K)-regular LDPC-BC ensemble as [LSCZ10].

[LSCZ10] M. Lentmaier, A. Sridharan, D. J. Costello, Jr., and K.Sh. Zigangirov, “Iterative decoding thresholdanalysis for LDPC convolutional codes,” IEEE Trans. Inf. Theory, 56:10, Oct. 2010.

29

Page 93: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

When symbols are perfectly known (BEC), all adjacent edges can be removedfrom the Tanner graph.

Why are SC-LDPC Codes Better?

...

The threshold saturates (converges) to a fixed value numericallyindistinguishable from the maximum a posteriori (MAP) threshold of the (J, K)-regular LDPC-BC ensemble as [LSCZ10].

[LSCZ10] M. Lentmaier, A. Sridharan, D. J. Costello, Jr., and K.Sh. Zigangirov, “Iterative decoding thresholdanalysis for LDPC convolutional codes,” IEEE Trans. Inf. Theory, 56:10, Oct. 2010.

For a more random-like ensemble, this has been proven analytically, first forthe BEC [KRU11], then for all BMS channels [KRU13].

[KRU11] S. Kudekar, T. J. Richardson and R. Urbanke, “Threshold saturation via spatial coupling: whyconvolutional LDPC ensembles perform so well over the BEC”, IEEE Trans. on Inf. Theory, 57:2, 2011

[KRU13] S. Kudekar, T. J. Richardson and R. Urbanke, “Spatially coupled ensembles universally achievecapacity under belief propagation”, IEEE Trans. on Inf. Theory, 59:12, 2013.

29

Page 94: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Threshold Saturation (BEC)

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

10-4

10-2

100

epsilon

Bit

era

sure

rate

BP threshold

MAPthreshold

(3,6)(3,6) BC

30

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

Page 95: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Threshold Saturation (BEC)

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

10-4

10-2

100

epsilon

Bit

era

sure

rate

(3,6)

MAP MAPBP BP

(3,6) BC

30

(5,10) BC

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

Page 96: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Threshold Saturation (BEC)

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

10-4

10-2

100

epsilon

Bit

era

sure

rate

(3,6) BC

MAP MAPBP BP

SC-LDPC codes

30

(5,10) BC

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

Page 97: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Threshold Saturation (BEC)

0.32 0.34 0.36 0.38 0.4 0.42 0.44 0.46 0.48 0.5

10-4

10-2

100

epsilon

Bit

era

sure

rate

(3,6) BC

MAP MAPBP BP

SC-LDPC codes

30

(5,10) BC

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

optimal decoding performance with a suboptimal iterative algorithm!

Page 98: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Threshold Saturation (AWGNC)

BPMAP

(3,6)-regularblock code

capa

city

~0.5dB

31

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

Page 99: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Threshold Saturation (AWGNC)

BPMAP

(3,6)-regularblock code

(4,8)-regularblock code

BPMAP

capa

city

~1.25dB

31

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

Page 100: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Threshold Saturation (AWGNC)

BPMAP

(3,6)-regularblock code

(4,8)-regularblock code

BPMAP

capa

city

spatially coupledcodes

(3,6)(4,8)

31

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

Page 101: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Threshold Saturation (AWGNC)

optimal decoding performance with a suboptimal iterative algorithm!

BPMAP

(3,6)-regularblock code

(4,8)-regularblock code

BPMAP

capa

city

spatially coupledcodes

(3,6)(4,8)

31

BP = iterative (suboptimal) decoding thresholdMAP = (optimal) maximum a posteriori threshold

Page 102: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

BEC Thresholds vs Distance Growth

By increasing J and K, we obtain capacity achieving (J,K)-regular SC-LDPC code ensembles with linear minimum distance growth.

32

-BC

(J,K)-regular SC-LDPC codes combine the best features of irregularand regular LDPC-BCs, i.e., capacity approaching thresholds and lineardistance growth.

Page 103: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

AWGNC Thresholds vs. Distance Growth

[MLC10] D. G. M. Mitchell, M. Lentmaier and D. J. Costello, Jr., “AWGN Channel Analysis of TerminatedLDPC Convolutional Codes”, Proc. Information Theory and Applications Workshop, San Diego, Feb. 2011.

Similar results are obtained for the AWGNC

33

Page 104: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Distance Measures for SC-LDPC Codes

As the minimum distance growth rates of terminated SC-LDPCcode ensembles tend to zero. However, the free distance growth rates ofthe unterminated ensembles remain constant.

34

(3,6)-regular unterminated SC-LDPC free distance growth rates

(3,6)-regular terminated SC-LDPCminimum distance growth rates

Page 105: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Distance Measures for SC-LDPC Codes

As the minimum distance growth rates of terminated SC-LDPCcode ensembles tend to zero. However, the free distance growth rates ofthe unterminated ensembles remain constant.

34

For large L, thestrength ofunterminatedensembles scaleswith theconstraint length and isindependent of L.

independent of L

normalized by L

(3,6)-regular unterminated SC-LDPC free distance growth rates

(3,6)-regular terminated SC-LDPCminimum distance growth rates

Page 106: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Distance Measures for SC-LDPC Codes

As the minimum distance growth rates of terminated SC-LDPCcode ensembles tend to zero. However, the free distance growth rates ofthe unterminated ensembles remain constant.

34

For large L, thestrength ofunterminatedensembles scaleswith theconstraint length and isindependent of L.An appropriatedistance measurefor 'convolutional-like' terminatedensembles shouldbe independent of L.

independent of L

normalized by L

(3,6)-regular unterminated SC-LDPC free distance growth rates

(3,6)-regular terminated SC-LDPCminimum distance growth rates

Page 107: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Outline

LDPC Block Codes

Spatially Coupled LDPC Codes

Protograph representation, edge-spreading construction, termination

Iterative decoding thresholds, threshold saturation, minimum distance

Practical Considerations

35

Parity-check matrix and Tanner graph representations, minimumdistance bounds, iterative decoding thresholds, protograph-based constructions

Window decoding; performance, latency, and complexity comparisonsto LDPC block codes; rate-compatibility; implementation aspects

Page 108: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

.

36

Block Decoding of SC-LDPC Codes m

ess

age

s pa

ssed

SC-LDPC codes can be decoded with standard iterative decoding schedules.

Page 109: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

.

36

Block Decoding of SC-LDPC Codes

Reliable messagesfrom the endspropagate throughthe graph toward thecenter as iterationsproceed.

me

ssag

es

pass

ed

SC-LDPC codes can be decoded with standard iterative decoding schedules.

Page 110: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

.

36

Block Decoding of SC-LDPC Codes

Reliable messagesfrom the endspropagate throughthe graph toward thecenter as iterationsproceed.

me

ssag

es

pass

ed

SC-LDPC codes can be decoded with standard iterative decoding schedules.

Page 111: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

.

36

Block Decoding of SC-LDPC Codes

Reliable messagesfrom the endspropagate throughthe graph toward thecenter as iterationsproceed.

me

ssag

es

pass

ed

SC-LDPC codes can be decoded with standard iterative decoding schedules.

Page 112: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

.

36

Block Decoding of SC-LDPC Codes

Reliable messagesfrom the endspropagate throughthe graph toward thecenter as iterationsproceed.

me

ssag

es

pass

ed

The frame error rate (FER) of a terminated graph can be analyzed

...

1 frame = L sections

(FER )The FER depends on L

SC-LDPC codes can be decoded with standard iterative decoding schedules.

Page 113: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 37

Block Decoding Performance

Consider LDPC-BCs and SC-LDPC codes with increasing frame length N

As N increasesthe LDPC-BCperformanceapproaches

Page 114: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 37

Block Decoding Performance

Consider LDPC-BCs and SC-LDPC codes with increasing frame length N

As N increasesthe LDPC-BCperformanceapproaches

Page 115: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 37

Block Decoding Performance

Consider LDPC-BCs and SC-LDPC codes with increasing frame length N

As N increasesthe LDPC-BCperformanceapproaches

Page 116: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 37

Block Decoding Performance

Consider LDPC-BCs and SC-LDPC codes with increasing frame length N

As N increasesthe LDPC-BCperformanceapproaches

As N increasesthe SC-LDPCcode performanceapproaches

outperformingthe LDPC-BCfor a sufficientlylarge N

Page 117: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 37

Block Decoding Performance

Consider LDPC-BCs and SC-LDPC codes with increasing frame length N

As N increasesthe LDPC-BCperformanceapproaches

As N increasesthe SC-LDPCcode performanceapproaches

outperformingthe LDPC-BCfor a sufficientlylarge N

Page 118: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 37

Block Decoding Performance

Consider LDPC-BCs and SC-LDPC codes with increasing frame length N

As N increasesthe LDPC-BCperformanceapproaches

As N increasesthe SC-LDPCcode performanceapproaches

outperformingthe LDPC-BCfor a sufficientlylarge N

Page 119: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 38

Window Decoding of SC-LDPC Codes

The highly localized (convolutional) structure is well-suited for efficientdecoding schedules that reduce memory and latency requirements. .

Page 120: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 38

Window Decoding of SC-LDPC Codes

Sliding window decoding (WD) updates nodes onlywithin a localized windowand then the window shiftsacross the graph [Lentmaieret al '10, Iyengar et al '12].

The highly localized (convolutional) structure is well-suited for efficientdecoding schedules that reduce memory and latency requirements.

width W

.

Page 121: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 38

Window Decoding of SC-LDPC Codes

Sliding window decoding (WD) updates nodes onlywithin a localized windowand then the window shiftsacross the graph [Lentmaieret al '10, Iyengar et al '12].

The highly localized (convolutional) structure is well-suited for efficientdecoding schedules that reduce memory and latency requirements.

width W

.

Page 122: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 38

Window Decoding of SC-LDPC Codes

Sliding window decoding (WD) updates nodes onlywithin a localized windowand then the window shiftsacross the graph [Lentmaieret al '10, Iyengar et al '12].

The highly localized (convolutional) structure is well-suited for efficientdecoding schedules that reduce memory and latency requirements.

width W

.

Page 123: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 38

Window Decoding of SC-LDPC Codes

Sliding window decoding (WD) updates nodes onlywithin a localized windowand then the window shiftsacross the graph [Lentmaieret al '10, Iyengar et al '12].

The highly localized (convolutional) structure is well-suited for efficientdecoding schedules that reduce memory and latency requirements.

width W

.

...One block of cM targetsymbols is decoded in eachwindow position

W

targetsymbols

Page 124: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44 38

Window Decoding of SC-LDPC Codes

Sliding window decoding (WD) updates nodes onlywithin a localized windowand then the window shiftsacross the graph [Lentmaieret al '10, Iyengar et al '12].

The highly localized (convolutional) structure is well-suited for efficientdecoding schedules that reduce memory and latency requirements.

width W

.

...One block of cM targetsymbols is decoded in eachwindow position

The window then shifts tothe right

targetsymbols

W

Page 125: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Window Decoding Performance

[LPF11] M. Lentmaier, M. M. Prenda, and G. Fettweis, “Efficient Message Passing Scheduling forTerminated LDPC Convolutional Codes”, Proc. IEEE ISIT, St. Petersburg, Russia, July 2011.

Latencies:LDPC:SC-LDPC:

39

Page 126: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Window Decoding Performance

[LPF11] M. Lentmaier, M. M. Prenda, and G. Fettweis, “Efficient Message Passing Scheduling forTerminated LDPC Convolutional Codes”, Proc. IEEE ISIT, St. Petersburg, Russia, July 2011.

Latencies:LDPC:SC-LDPC:

For equal liftingfactors, SC-LDPCcodes display alarge convolutionalgain at the cost ofincreased latency.

convolutional gain

39

Page 127: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Window Decoding Performance

[LPF11] M. Lentmaier, M. M. Prenda, and G. Fettweis, “Efficient Message Passing Scheduling forTerminated LDPC Convolutional Codes”, Proc. IEEE ISIT, St. Petersburg, Russia, July 2011.

Latencies:LDPC:SC-LDPC:

For equal liftingfactors, SC-LDPCcodes display alarge convolutionalgain at the cost ofincreased latency.

For equal latency,SC-LDPC codesstill display asignificantperformance gain.

equallatency

39

Page 128: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Window Decoding Performance

[LPF11] M. Lentmaier, M. M. Prenda, and G. Fettweis, “Efficient Message Passing Scheduling forTerminated LDPC Convolutional Codes”, Proc. IEEE ISIT, St. Petersburg, Russia, July 2011.

Latencies:LDPC:SC-LDPC:

For equal liftingfactors, SC-LDPCcodes display alarge convolutionalgain at the cost ofincreased latency.

For equal latency,SC-LDPC codesstill display asignificantperformance gain.

equallatency

39

Page 129: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Window Decoding Performance

[LPF11] M. Lentmaier, M. M. Prenda, and G. Fettweis, “Efficient Message Passing Scheduling forTerminated LDPC Convolutional Codes”, Proc. IEEE ISIT, St. Petersburg, Russia, July 2011.

Latencies:LDPC:SC-LDPC:

For equal liftingfactors, SC-LDPCcodes display alarge convolutionalgain at the cost ofincreased latency.

For equal latency,SC-LDPC codesstill display asignificantperformance gain.

equallatency

39

Trade-off in M vs W

Page 130: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases. does not decreasesignificantly beyond acertain W

Equal Latency Comparison for (3,6)-Regular LDPC Codes

BC

SC

Latencies:LDPC:SC-LDPC:

40

Page 131: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases. does not decreasesignificantly beyond acertain W

Equal Latency Comparison for (3,6)-Regular LDPC Codes

BC

SC

SC

Latencies:LDPC:SC-LDPC:

40

Page 132: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases. does not decreasesignificantly beyond acertain W

Equal Latency Comparison for (3,6)-Regular LDPC Codes

BC

SC

SC

SC

Latencies:LDPC:SC-LDPC:

40

Page 133: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases. does not decreasesignificantly beyond acertain W

Equal Latency Comparison for (3,6)-Regular LDPC Codes

BC

SC

SC

SC

SC

Latencies:LDPC:SC-LDPC:

40

Page 134: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases. does not decreasesignificantly beyond acertain W

Equal Latency Comparison for (3,6)-Regular LDPC Codes

BC

SC

SC

SC

SC

SC

Latencies:LDPC:SC-LDPC:

40

Page 135: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases. does not decreasesignificantly beyond acertain W

Equal Latency Comparison for (3,6)-Regular LDPC Codes

BC

SC

SC

SC

SC

SC

SC

Latencies:LDPC:SC-LDPC:

40

Page 136: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Required to achieve a BER of as a function of latency:

decreases as W (and thus thelatency) increases. does not decreasesignificantly beyond acertain W

Equal Latency Comparison for (3,6)-Regular LDPC Codes

large improvescode performance.

large W improvesdecoder performance.

When choosingparameters:

BC

SC

SC

SC

SC

SC

SC

Latencies:LDPC:SC-LDPC:

40

Page 137: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Regular SC-LDPC Codes vs. Irregular LDPC-BCs

Consider a comparison of a (3,6)-regular SC-LDPC code vs. anirregular-repeat-accumulate (IRA) LDPC-BC with optimizedprotograph taken from the WiMAX standard

41

Ex:

Page 138: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Regular SC-LDPC Codes vs. Irregular LDPC-BCs

Consider a comparison of a (3,6)-regular SC-LDPC code vs. anirregular-repeat-accumulate (IRA) LDPC-BC with optimizedprotograph taken from the WiMAX standard

The IRA LDPC-BC ensemble has rate R=0.5, BEC threshold , and AWGNC threshold dB.

41

Ex:

Page 139: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Regular SC-LDPC Codes vs. Irregular LDPC-BCs

Consider a comparison of a (3,6)-regular SC-LDPC code vs. anirregular-repeat-accumulate (IRA) LDPC-BC with optimizedprotograph taken from the WiMAX standard

The IRA LDPC-BC ensemble has rate R=0.5, BEC threshold , and AWGNC threshold dB.

We compare this to a (3,6)-regular SC-LDPC code ensemble withL=50, R=0.49, and thresholds and dB.

41

Ex:

Page 140: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Regular SC-LDPC Codes vs. Irregular LDPC-BCs

Consider a comparison of a (3,6)-regular SC-LDPC code vs. anirregular-repeat-accumulate (IRA) LDPC-BC with optimizedprotograph taken from the WiMAX standard

The IRA LDPC-BC ensemble has rate R=0.5, BEC threshold , and AWGNC threshold dB.

We compare this to a (3,6)-regular SC-LDPC code ensemble withL=50, R=0.49, and thresholds and dB.

For the SC-LDPC code, we choose W=6 and M=500 so that thelatency of both codes is 6000 bits. (Since a code symbol is present inW=6 'windows', we allow fewer iterations per position for the SC-LDPC window decoder.)

41

Ex:

Page 141: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Gaps tothreshold willreduce withincreasinglatency

42

Regular SC-LDPC Codes vs. Irregular LDPC-BCs

Page 142: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Gaps tothreshold willreduce withincreasinglatency

Theasymptoticallygood regularSC-LDPC codeshows no sign ofan error floor

42

Regular SC-LDPC Codes vs. Irregular LDPC-BCs

Page 143: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Gaps tothreshold willreduce withincreasinglatency

Theasymptoticallygood regularSC-LDPC codeshows no sign ofan error floor

The regular SC-LDPC codestructure hasimplementationadvantages

42

Regular SC-LDPC Codes vs. Irregular LDPC-BCs

Page 144: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Implementation Aspects

As a result of their capacity approaching performance and simplestructure, regular SC-LDPC codes may be attractive for future codingstandards. Several key features will require further investigation:

Hardware advantages of QC designs obtained by circulant liftings

Hardware advantages of the 'asymptotically-regular' structure

Design advantages of flexible frame length and flexible rateobtained by varying M, L, and/or puncturing

43

Page 145: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Implementation Aspects

As a result of their capacity approaching performance and simplestructure, regular SC-LDPC codes may be attractive for future codingstandards. Several key features will require further investigation:

Hardware advantages of QC designs obtained by circulant liftings

Hardware advantages of the 'asymptotically-regular' structure

Design advantages of flexible frame length and flexible rateobtained by varying M, L, and/or puncturing

Of particular importance for applications requiring extremely lowdecoded bit error rates (e.g., optical communication, data storage) isan investigation of error floor issues related to stopping sets,trapping sets, and absorbing sets.

43

Page 146: Daniel J. Costello, Jr. - ece.engin.umich.edu · Daniel J. Costello, Jr. Dept. of Electrical Engineering, Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind? University

D. J. Costello, Jr., “Spatially Coupled LDPC Codes: Is This What Shannon Had In Mind?” / 44

Conclusions

Spatially coupled LDPC code ensembles achieve threshold saturation,i.e., their iterative decoding thresholds (for large L and M) approach theMAP decoding thresholds of the underlying LDPC block code ensembles.

The threshold saturation and linear minimum distance growth propertiesof (J,K)-regular SC-LDPC codes combine the best asymptotic featuresof both regular and irregular LDPC-BCs.

With window decoding, SC-LDPC codes also compare favorably toLDPC-BCs in the finite-length regime, providing flexible tradeoffsbetween BER performance, decoding latency, and decoding complexity.

44

SC-LDPC codes can be punctured to achieve robustly good performanceover a wide variety of code rates.