polar coding - status and prospectskedart/ee376a_winter1617/...(uniform) wvec vector channel combine...
TRANSCRIPT
Polarization Polar coding Performance
Polar CodingStatus and Prospects
Erdal Arıkan
Electrical-Electronics Engineering DepartmentBilkent UniversityAnkara, Turkey
1 August 2011The IEEE International Symposium on Information Theory
ISIT’2011Saint Petersburg, Russia
Polarization Polar coding Performance
The channel
Let W : X → Y be a binary-input discrete memoryless channel
WX Y
I input alphabet: X = {0, 1},
I output alphabet: Y,
I transition probabilities:
W (y |x), x ∈ X , y ∈ Y
Polarization Polar coding Performance
The channel
Let W : X → Y be a binary-input discrete memoryless channel
WX Y
I input alphabet: X = {0, 1},
I output alphabet: Y,
I transition probabilities:
W (y |x), x ∈ X , y ∈ Y
Polarization Polar coding Performance
The channel
Let W : X → Y be a binary-input discrete memoryless channel
WX Y
I input alphabet: X = {0, 1},
I output alphabet: Y,
I transition probabilities:
W (y |x), x ∈ X , y ∈ Y
Polarization Polar coding Performance
The channel
Let W : X → Y be a binary-input discrete memoryless channel
WX Y
I input alphabet: X = {0, 1},
I output alphabet: Y,
I transition probabilities:
W (y |x), x ∈ X , y ∈ Y
Polarization Polar coding Performance
Symmetry assumption
Assume that the channel has “input-output symmetry.”
Polarization Polar coding Performance
Symmetry assumption
Assume that the channel has “input-output symmetry.”
Examples:
1− ε
1− ε
ε
ε
1
0
1
0
BSC(ε)
Polarization Polar coding Performance
Symmetry assumption
Assume that the channel has “input-output symmetry.”
Examples:
1− ε
1− ε
ε
ε
1
0
1
0
BSC(ε)
1− ε
1− ε
ε
ε
1
0
1
0
?
BEC(ε)
Polarization Polar coding Performance
Capacity
For channels with input-output symmetry, the capacity is given by
C (W )∆= I (X ;Y ), with X ∼ unif. {0, 1}
Polarization Polar coding Performance
Capacity
For channels with input-output symmetry, the capacity is given by
C (W )∆= I (X ;Y ), with X ∼ unif. {0, 1}
Use base-2 logarithms:
0 ≤ C (W ) ≤ 1
Polarization Polar coding Performance
The main idea
I Channel coding problem trivial for two types of channelsI Perfect: C (W ) = 1I Useless: C (W ) = 0
I Transform ordinary W into such extreme channels
Polarization Polar coding Performance
The main idea
I Channel coding problem trivial for two types of channelsI Perfect: C (W ) = 1I Useless: C (W ) = 0
I Transform ordinary W into such extreme channels
Polarization Polar coding Performance
The main idea
I Channel coding problem trivial for two types of channelsI Perfect: C (W ) = 1I Useless: C (W ) = 0
I Transform ordinary W into such extreme channels
Polarization Polar coding Performance
The main idea
I Channel coding problem trivial for two types of channelsI Perfect: C (W ) = 1I Useless: C (W ) = 0
I Transform ordinary W into such extreme channels
Polarization Polar coding Performance
The method: aggregate and redistribute capacity
W
W
b
b
b
W
Original channels(uniform)
Polarization Polar coding Performance
The method: aggregate and redistribute capacity
W
W
b
b
b
W
Original channels(uniform)
Wvec
Vectorchannel
Combine
Polarization Polar coding Performance
The method: aggregate and redistribute capacity
W
W
b
b
b
W
Original channels(uniform)
Wvec
Vectorchannel
Combine
WN
WN−1
b
b
b
W1
Split
New channels(polarized)
Polarization Polar coding Performance
Combining
I Begin with N copies of W ,
I use a 1-1 mapping
GN : {0, 1}N → {0, 1}N
I to create a vector channel
Wvec : UN → Y N
W
W
WXN
X2
X1
YN
Y2
Y1
Polarization Polar coding Performance
Combining
I Begin with N copies of W ,
I use a 1-1 mapping
GN : {0, 1}N → {0, 1}N
I to create a vector channel
Wvec : UN → Y N
W
W
WXN
X2
X1
YN
Y2
Y1
GN
UN
U2
U1
Polarization Polar coding Performance
Combining
I Begin with N copies of W ,
I use a 1-1 mapping
GN : {0, 1}N → {0, 1}N
I to create a vector channel
Wvec : UN → Y N
W
W
WXN
X2
X1
YN
Y2
Y1
GN
UN
U2
U1
Wvec
Polarization Polar coding Performance
Conservation of capacity
Combining operation is lossless:
I Take U1, . . . ,UN i.i.d. unif. {0, 1}
I then, X1, . . . ,XN i.i.d. unif. {0, 1}
I and
C (Wvec) = I (UN ;Y N)
= I (XN ;Y N)
= NC (W )
W
W
W
GN
XN
X2
X1
YN
Y2
Y1
UN
U2
U1
Wvec
Polarization Polar coding Performance
Conservation of capacity
Combining operation is lossless:
I Take U1, . . . ,UN i.i.d. unif. {0, 1}
I then, X1, . . . ,XN i.i.d. unif. {0, 1}
I and
C (Wvec) = I (UN ;Y N)
= I (XN ;Y N)
= NC (W )
W
W
W
GN
XN
X2
X1
YN
Y2
Y1
UN
U2
U1
Wvec
Polarization Polar coding Performance
Conservation of capacity
Combining operation is lossless:
I Take U1, . . . ,UN i.i.d. unif. {0, 1}
I then, X1, . . . ,XN i.i.d. unif. {0, 1}
I and
C (Wvec) = I (UN ;Y N)
= I (XN ;Y N)
= NC (W )
W
W
W
GN
XN
X2
X1
YN
Y2
Y1
UN
U2
U1
Wvec
Polarization Polar coding Performance
Splitting
C (Wvec) = I (UN ;Y N)
Wvec
UN
Ui+1
Ui
Ui−1
U1
YN
Yi
Y1
Polarization Polar coding Performance
Splitting
C (Wvec) = I (UN ;Y N)
=N∑
i=1
I (Ui ;YN ,U i−1)
Wvec
UN
Ui+1
Ui
Ui−1
U1
YN
Yi
Y1
Polarization Polar coding Performance
Splitting
C (Wvec) = I (UN ;Y N)
=N∑
i=1
I (Ui ;YN ,U i−1)
Define bit-channels
Wi : Ui → (Y N ,U i−1)
Wvec
UN
Ui+1
Ui
Ui−1
U1
U1
Ui−1
YN
Yi
Y1
Wi
Polarization Polar coding Performance
Splitting
C (Wvec) = I (UN ;Y N)
=N∑
i=1
I (Ui ;YN ,U i−1)
=N∑
i=1
C (Wi )
Define bit-channels
Wi : Ui → (Y N ,U i−1)
Wvec
UN
Ui+1
Ui
Ui−1
U1
U1
Ui−1
YN
Yi
Y1
Wi
Polarization Polar coding Performance
Polarization is commonplace
I Polarization is the rule not theexception
I A random permutation
GN : {0, 1}N → {0, 1}N
is a good polarizer with highprobability
I Equivalent to Shannon’s randomcoding approach
W
W
W
GN
XN
X2
X1
YN
Y2
Y1
UN
U2
U1
Polarization Polar coding Performance
Polarization is commonplace
I Polarization is the rule not theexception
I A random permutation
GN : {0, 1}N → {0, 1}N
is a good polarizer with highprobability
I Equivalent to Shannon’s randomcoding approach
W
W
W
GN
XN
X2
X1
YN
Y2
Y1
UN
U2
U1
Polarization Polar coding Performance
Polarization is commonplace
I Polarization is the rule not theexception
I A random permutation
GN : {0, 1}N → {0, 1}N
is a good polarizer with highprobability
I Equivalent to Shannon’s randomcoding approach
W
W
W
GN
XN
X2
X1
YN
Y2
Y1
UN
U2
U1
Polarization Polar coding Performance
Random polarizers: stepwise, isotropic
5 10 15 20 25 300
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Polarization Polar coding Performance
Random polarizers: stepwise, isotropic
5 10 15 20 25 300
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Isotropy: any redistribution order is as good as any other.
Polarization Polar coding Performance
The complexity issue
I Random polarizers lack structure, too complex to implement
I Need a low-complexity polarizer
I May sacrifice stepwise, isotropic properties of randompolarizers in return for less complexity
Polarization Polar coding Performance
The complexity issue
I Random polarizers lack structure, too complex to implement
I Need a low-complexity polarizer
I May sacrifice stepwise, isotropic properties of randompolarizers in return for less complexity
Polarization Polar coding Performance
The complexity issue
I Random polarizers lack structure, too complex to implement
I Need a low-complexity polarizer
I May sacrifice stepwise, isotropic properties of randompolarizers in return for less complexity
Polarization Polar coding Performance
Basic module for a low-complexity scheme
Combine two copies of W
W
W
Y2
Y1
X2
X1
Polarization Polar coding Performance
Basic module for a low-complexity scheme
Combine two copies of W
+
U2
U1
G2
W
W
Y2
Y1
X2
X1
Polarization Polar coding Performance
Basic module for a low-complexity scheme
Combine two copies of W
+
U2
U1
G2
W
W
Y2
Y1
X2
X1
and split to create two bit-channels
W1 : U1 → (Y1,Y2)
W2 : U2 → (Y1,Y2,U1)
Polarization Polar coding Performance
The first bit-channel W1
W1 : U1 → (Y1,Y2)
+
random U2
U1
W
W
Y2
Y1
Polarization Polar coding Performance
The first bit-channel W1
W1 : U1 → (Y1,Y2)
+
random U2
U1
W
W
Y2
Y1
C (W1) = I (U1;Y1,Y2)
Polarization Polar coding Performance
The second bit-channel W2
W2 : U2 → (Y1,Y2,U1)
+
U2
U1
W
W
Y2
Y1
Polarization Polar coding Performance
The second bit-channel W2
W2 : U2 → (Y1,Y2,U1)
+
U2
U1
W
W
Y2
Y1
C (W2) = I (U2;Y1,Y2,U1)
Polarization Polar coding Performance
Capacity conserved but redistributed unevenly
+
U2
U1
W
W
Y2
Y1
X2
X1
I Conservation:
C (W1) + C (W2) = 2C (W )
I Extremization:
C (W1) ≤ C (W ) ≤ C (W2)
with equality iff C (W ) equals 0 or 1.
Polarization Polar coding Performance
Capacity conserved but redistributed unevenly
+
U2
U1
W
W
Y2
Y1
X2
X1
I Conservation:
C (W1) + C (W2) = 2C (W )
I Extremization:
C (W1) ≤ C (W ) ≤ C (W2)
with equality iff C (W ) equals 0 or 1.
Polarization Polar coding Performance
Notation
The two channels created by the basic transform
(W ,W ) → (W1,W2)
will be denoted also as
W− = W1 and W+ = W2
Polarization Polar coding Performance
Notation
The two channels created by the basic transform
(W ,W ) → (W1,W2)
will be denoted also as
W− = W1 and W+ = W2
Likewise, we write W−−, W−+ for descendants of W−; and W+−,W++ for descendants of W+.
Polarization Polar coding Performance
For the size-4 construction
+
W
W
Polarization Polar coding Performance
... duplicate the basic transform
+
+
W
W
W
W
Polarization Polar coding Performance
... obtain a pair of W − and W+ each
W+
W+
W−
W−
Polarization Polar coding Performance
... apply basic transform on each pair
+
+
W+
W+
W−
W−
Polarization Polar coding Performance
... decode in the indicated order
+
+
W+
W+
W−
W−
U4
U2
U3
U1
Polarization Polar coding Performance
... obtain the four new bit-channels
W++
W−+
W+−
W−−
U4
U2
U3
U1
Polarization Polar coding Performance
Overall size-4 construction
+
+
+
+
W
W
W
W
U4
U2
U3
U1
Y4
Y2
Y3
Y1
X4
X2
X3
X1
Polarization Polar coding Performance
“Rewire” for standard-form size-4 construction
+
+
+
+
W
W
W
W
U4
U3
U2
U1
Y4
Y3
Y2
Y1
X4
X3
X2
X1
Polarization Polar coding Performance
Size 8 construction
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
U5
U4
U3
U2
U1
X8
X7
X6
X5
X4
X3
X2
X1
Polarization Polar coding Performance
Demonstration of polarization
Polarization is easy to analyze when W is a BEC.
If W is a BEC(ε), then so are W−
and W+, with erasure probabili-ties
ε−∆= 2ε− ε2
andε+
∆= ε2
respectively.1− ε
1− ε
ε
ε
1
0
1
0
?
W
Polarization Polar coding Performance
Demonstration of polarization
Polarization is easy to analyze when W is a BEC.
If W is a BEC(ε), then so are W−
and W+, with erasure probabili-ties
ε−∆= 2ε− ε2
andε+
∆= ε2
respectively.1− ε−
1− ε−
ε−
ε−
1
0
1
0
?
W−
Polarization Polar coding Performance
Demonstration of polarization
Polarization is easy to analyze when W is a BEC.
If W is a BEC(ε), then so are W−
and W+, with erasure probabili-ties
ε−∆= 2ε− ε2
andε+
∆= ε2
respectively.1− ε+
1− ε+
ε+
ε+
1
0
1
0
?
W+
Polarization Polar coding Performance
Polarization for BEC(12): N = 16
2 4 6 8 10 12 14 160
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Capacity of bit channels
N=16
Polarization Polar coding Performance
Polarization for BEC(12): N = 32
5 10 15 20 25 300
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Capacity of bit channels
N=32
Polarization Polar coding Performance
Polarization for BEC(12): N = 64
10 20 30 40 50 600
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Capacity of bit channels
N=64
Polarization Polar coding Performance
Polarization for BEC(12): N = 128
20 40 60 80 100 1200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Capacity of bit channels
N=128
Polarization Polar coding Performance
Polarization for BEC(12): N = 256
50 100 150 200 2500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Capacity of bit channels
N=256
Polarization Polar coding Performance
Polarization for BEC(12): N = 512
50 100 150 200 250 300 350 400 450 5000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Capacity of bit channels
N=512
Polarization Polar coding Performance
Polarization for BEC(12): N = 1024
100 200 300 400 500 600 700 800 900 10000
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Bit channel index
Cap
acity
Capacity of bit channels
N=1024
Polarization Polar coding Performance
Polarization martingale
0
1
C(W )
Polarization Polar coding Performance
Polarization martingale
0
1
1
C(W )
C(W2)
C(W1)
Polarization Polar coding Performance
Polarization martingale
0
1
1 22
C(W )
C(W2)
C(W1)
C(W++)
C(W−+)
C(W+−)
C(W−−)
Polarization Polar coding Performance
Polarization martingale
0
1
1 22
C(W )
C(W2)
C(W1)
C(W++)
C(W−+)
C(W+−)
C(W−−)
Polarization Polar coding Performance
Polarization martingale
0
1
1 22 3333
C(W )
C(W2)
C(W1)
C(W++)
C(W−+)
C(W+−)
C(W−−)
Polarization Polar coding Performance
Polarization martingale
0
1
1 22 3333 44444444
C(W )
C(W2)
C(W1)
C(W++)
C(W−+)
C(W+−)
C(W−−)
Polarization Polar coding Performance
Polarization martingale
0
1
1 22 3333 44444444 5555555555555555
C(W )
C(W2)
C(W1)
C(W++)
C(W−+)
C(W+−)
C(W−−)
Polarization Polar coding Performance
Polarization martingale
0
1
1 22 3333 44444444 5555555555555555 66666666666666666666666666666666
C(W )
C(W2)
C(W1)
C(W++)
C(W−+)
C(W+−)
C(W−−)
Polarization Polar coding Performance
Polarization martingale
0
1
1 22 3333 44444444 5555555555555555 66666666666666666666666666666666 7777777777777777777777777777777777777777777777777777777777777777
C(W )
C(W2)
C(W1)
C(W++)
C(W−+)
C(W+−)
C(W−−)
Polarization Polar coding Performance
Polarization martingale
0
1
1 22 3333 44444444 5555555555555555 66666666666666666666666666666666 7777777777777777777777777777777777777777777777777777777777777777 88888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888
C(W )
C(W2)
C(W1)
C(W++)
C(W−+)
C(W+−)
C(W−−)
Polarization Polar coding Performance
Theorem (Polarization, A. 2007)
The bit-channel capacities {C (Wi )} polarize: for any
δ ∈ (0, 1), as the construction size N grows
[
no. channels with C (Wi ) > 1− δ
N
]
−→ C (W )
and[
no. channels with C (Wi ) < δ
N
]
−→ 1− C (W )
Theorem (Rate of polarization, A. and Telatar (2008))
Above theorem holds with δ ≈ 2−√
N . 0
δ
1− δ
1
Polarization Polar coding Performance
Theorem (Polarization, A. 2007)
The bit-channel capacities {C (Wi )} polarize: for any
δ ∈ (0, 1), as the construction size N grows
[
no. channels with C (Wi ) > 1− δ
N
]
−→ C (W )
and[
no. channels with C (Wi ) < δ
N
]
−→ 1− C (W )
Theorem (Rate of polarization, A. and Telatar (2008))
Above theorem holds with δ ≈ 2−√
N . 0
δ
1− δ
1
Polarization Polar coding Performance
Polar code example: W = BEC(12), N = 8, rate 1/2
C(Wi )
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
U5
U4
U3
U2
U1
Polarization Polar coding Performance
Polar code example: W = BEC(12), N = 8, rate 1/2
C(Wi )
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
Rank
8
7
6
4
5
3
2
1
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
U5
U4
U3
U2
U1
Polarization Polar coding Performance
Polar code example: W = BEC(12), N = 8, rate 1/2
C(Wi )
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
Rank
8
7
6
4
5
3
2
1
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
U5
U4
U3
U2
U1
data
Polarization Polar coding Performance
Polar code example: W = BEC(12), N = 8, rate 1/2
C(Wi )
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
Rank
8
7
6
4
5
3
2
1
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
U5
U4
U3
U2
U1
data
data
Polarization Polar coding Performance
Polar code example: W = BEC(12), N = 8, rate 1/2
C(Wi )
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
Rank
8
7
6
4
5
3
2
1
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
U5
U4
U3
U2
U1
data
data
data
Polarization Polar coding Performance
Polar code example: W = BEC(12), N = 8, rate 1/2
C(Wi )
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
Rank
8
7
6
4
5
3
2
1
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
U5
U4
U3
U2
U1
data
data
data
data
Polarization Polar coding Performance
Polar code example: W = BEC(12), N = 8, rate 1/2
C(Wi )
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
Rank
8
7
6
4
5
3
2
1
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
U5
U4
U3
U2
U1
data
data
data
frozen
data
frozen
frozen
frozen
Polarization Polar coding Performance
Polar code example: W = BEC(12), N = 8, rate 1/2
C(Wi )
0.0039
0.1211
0.1914
0.6836
0.3164
0.8086
0.8789
0.9961
Rank
8
7
6
4
5
3
2
1
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
U8
U7
U6
0
U4
0
0
0
data
data
data
frozen
data
frozen
frozen
frozen
Polarization Polar coding Performance
Construction complexity
I An O(N) construction algorithm exists that usesdensity-evolution
I First proposed by Mori and Tanaka, without finite-precisionimplementation details
I Tal and Vardy introduced smart quantization methods for apractical implementation
I The algorithm works well in practice but a precise proof ofO(N) complexity still lacking
I Recent work: Pedarsani, Hassani, Tal, and Telatar (ISIT’2011)
Polarization Polar coding Performance
Construction complexity
I An O(N) construction algorithm exists that usesdensity-evolution
I First proposed by Mori and Tanaka, without finite-precisionimplementation details
I Tal and Vardy introduced smart quantization methods for apractical implementation
I The algorithm works well in practice but a precise proof ofO(N) complexity still lacking
I Recent work: Pedarsani, Hassani, Tal, and Telatar (ISIT’2011)
Polarization Polar coding Performance
Construction complexity
I An O(N) construction algorithm exists that usesdensity-evolution
I First proposed by Mori and Tanaka, without finite-precisionimplementation details
I Tal and Vardy introduced smart quantization methods for apractical implementation
I The algorithm works well in practice but a precise proof ofO(N) complexity still lacking
I Recent work: Pedarsani, Hassani, Tal, and Telatar (ISIT’2011)
Polarization Polar coding Performance
Construction complexity
I An O(N) construction algorithm exists that usesdensity-evolution
I First proposed by Mori and Tanaka, without finite-precisionimplementation details
I Tal and Vardy introduced smart quantization methods for apractical implementation
I The algorithm works well in practice but a precise proof ofO(N) complexity still lacking
I Recent work: Pedarsani, Hassani, Tal, and Telatar (ISIT’2011)
Polarization Polar coding Performance
Construction complexity
I An O(N) construction algorithm exists that usesdensity-evolution
I First proposed by Mori and Tanaka, without finite-precisionimplementation details
I Tal and Vardy introduced smart quantization methods for apractical implementation
I The algorithm works well in practice but a precise proof ofO(N) complexity still lacking
I Recent work: Pedarsani, Hassani, Tal, and Telatar (ISIT’2011)
Polarization Polar coding Performance
Encoding complexity
Encoding complexity for polar coding is O(N logN).
Polarization Polar coding Performance
Encoding: an example
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
1
0
1
0
1
0
0
0
free
free
free
frozen
free
frozen
frozen
frozen
Polarization Polar coding Performance
Encoding: an example
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
1
0
1
0
1
0
0
0
1
1
1
1
1
1
0
0
free
free
free
frozen
free
frozen
frozen
frozen
Polarization Polar coding Performance
Encoding: an example
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
1
0
1
0
1
0
0
0
1
1
1
1
1
1
0
0
1
1
0
0
1
1
1
1
free
free
free
frozen
free
frozen
frozen
frozen
Polarization Polar coding Performance
Encoding: an example
+
+
+
+
+
+
+
+
+
+
+
+
W
W
W
W
W
W
W
W
Y8
Y7
Y6
Y5
Y4
Y3
Y2
Y1
1
0
1
0
1
0
0
0
1
1
1
1
1
1
0
0
1
1
0
0
1
1
1
1
1
1
0
0
0
0
1
1
free
free
free
frozen
free
frozen
frozen
frozen
Polarization Polar coding Performance
Successive cancellation decoding complexity
(A. 2007)
Complexity of successive cancellation decoding for polar codes isO(N logN).
Polarization Polar coding Performance
Successive cancellation decoding complexity
(A. 2007)
Complexity of successive cancellation decoding for polar codes isO(N logN).
Earlier work on similar decoders:
I Kabatiansky (1990)
I Schnabl and Bossert (1996)
I Dumer and co-authors (from 1990s)
I Burnashev and Dumer (2006-2009)
Polarization Polar coding Performance
Performance of SC decoder
(A. and Telatar, 2008)
For any rate R < C (W ) and block-length N, the probability offrame error for polar codes under SC decoding is bounded roughlyas
Pe(N,R) = o(
2−√
N
)
I Prior result (A. 2007): Pe(N,R) = o(
N−1/4)
.
I Latest result: A rate-dependent refinement of the“square-root” bound has been given by Tanaka & Mori(2010) and Hassani & Urbanke (2010)
Polarization Polar coding Performance
Performance of SC decoder
(A. and Telatar, 2008)
For any rate R < C (W ) and block-length N, the probability offrame error for polar codes under SC decoding is bounded roughlyas
Pe(N,R) = o(
2−√
N
)
I Prior result (A. 2007): Pe(N,R) = o(
N−1/4)
.
I Latest result: A rate-dependent refinement of the“square-root” bound has been given by Tanaka & Mori(2010) and Hassani & Urbanke (2010)
Polarization Polar coding Performance
Performance of SC decoder
(A. and Telatar, 2008)
For any rate R < C (W ) and block-length N, the probability offrame error for polar codes under SC decoding is bounded roughlyas
Pe(N,R) = o(
2−√
N
)
I Prior result (A. 2007): Pe(N,R) = o(
N−1/4)
.
I Latest result: A rate-dependent refinement of the“square-root” bound has been given by Tanaka & Mori(2010) and Hassani & Urbanke (2010)
Polarization Polar coding Performance
Polar coding summary
Given W , N = 2n, and R < C (W ), a polar code with these param-eters has
I construction complexity O(N) (conjecture),
I encoding complexity ≈ N logN,
I decoding complexity ≈ N logN,
I and frame error probability Pe(N,R) ≈ o(2−√
N)
Polarization Polar coding Performance
Polar coding summary
Given W , N = 2n, and R < C (W ), a polar code with these param-eters has
I construction complexity O(N) (conjecture),
I encoding complexity ≈ N logN,
I decoding complexity ≈ N logN,
I and frame error probability Pe(N,R) ≈ o(2−√
N)
Polarization Polar coding Performance
Polar coding summary
Given W , N = 2n, and R < C (W ), a polar code with these param-eters has
I construction complexity O(N) (conjecture),
I encoding complexity ≈ N logN,
I decoding complexity ≈ N logN,
I and frame error probability Pe(N,R) ≈ o(2−√
N)
Polarization Polar coding Performance
Polar coding summary
Given W , N = 2n, and R < C (W ), a polar code with these param-eters has
I construction complexity O(N) (conjecture),
I encoding complexity ≈ N logN,
I decoding complexity ≈ N logN,
I and frame error probability Pe(N,R) ≈ o(2−√
N)
Polarization Polar coding Performance
Polar coding in other contexts
I Source coding (lossless)
I Source coding in the presence of memory
I Lossy source coding
I Slepian-Wolf problem
I Wyner-Ziv problem
I Gelfand-Pinsker problem
I MAC
I Degraded-broadcast channel
I Wyner wiretap channel
I Randomness extraction
I ...
Polarization Polar coding Performance
Channel-coding scenarios
Topic Ref.
Single-user q-ary channels Sasoglu, Telatar, A. (2009)Multi-access channels Sasoglu, Telatar, Yeh (2010)
m-user MAC Abbe and Telatar (2010)Wyner wiretap channel Mahdavifar and Vardy (2009)
′′ Hof and Shamai (2010)′′ Koyluoglu and El Gamal (2010)′′ Andersson et al. (2010)
Relay channel Andersson et al. (2010)′′ Blasco-Serrano et al. (2010)′′ Karzand (2011)
Compund channel coding Hassani, Korada, Urbanke (2009)
Polarization Polar coding Performance
Source-coding scenarios
Topic Ref.
Lossless source coding Hussami, Korada, Urbanke (2009)Rate-distortion coding Korada and Urbanke (2009)
q-ary lossless source coding Karzand and Telatar (2010)Direct source polarization A. (2010)Universal polar coding Abbe (2010)
Sparse recovery Abbe (2010)Randomness extraction Abbe (2011)
Ergodic source polarization Sasoglu (2011)
Polarization Polar coding Performance
Scenarios with side-information
Topic Ref.
Wyner-Ziv coding Korada and Urbanke (2009)Gelfand-Pinsker coding Korada and Urbanke (2009)Slepian-Wolf coding Hussami, Korada, Urbanke (2009)
Polarization Polar coding Performance
Generalized polarization schemesq: alphabet size`: dimension of basic transform (kernel)E : rate of polarization exponent
q ` Exponent E Kernel Ref.
2 2 to 15 ≤ 1/2 Any linear KSU (2009)2 16 0.51828 BCH KSU (2009)2 31 0.52643 BCH KSU (2009)4 4 0.573120 Reed-Solomon MT (2010)2 14 0.50193 Nonlinear PSL (2011)2 15 0.50773 Nonlinear PSL (2011)2 16 0.52742 Nonlinear PSL (2011)
KSU: Korada, Sasoglu, UrbankeMT: Mori and TanakaPSL: Presman, Shapira, Litsyn
Polarization Polar coding Performance
Performance comparison: Polar vs. Turbo
Turbo code
I WiMAX CTC
I Duobinary, memory 3
I QAM over AWGN channel
I Gray mapping
I BICM
I Simulator: “CodedModulation Library”
Polar code
I Standard construction
I Successive cancellationdecoding
I QAM over AWGN channel
I Natural mapping
I Multi-level PAM
I PAM over AWGN channel
Polarization Polar coding Performance
Example: 8-PAM as 3 bit channels
I PAM signals selected by three bits (b1, b2, b3)
I Three layers of binary channels created
I Each layer encoded independently
I Layers decoded in the order b3, b2, b1
Bit b1 0 1
-4 42-PAM
Polarization Polar coding Performance
Example: 8-PAM as 3 bit channels
I PAM signals selected by three bits (b1, b2, b3)
I Three layers of binary channels created
I Each layer encoded independently
I Layers decoded in the order b3, b2, b1
Bit b2 0 1 10
-6 -2 2 64-PAM
Bit b1 0 1
Polarization Polar coding Performance
Example: 8-PAM as 3 bit channels
I PAM signals selected by three bits (b1, b2, b3)
I Three layers of binary channels created
I Each layer encoded independently
I Layers decoded in the order b3, b2, b1
Bit b2 0 1 10
Bit b1 0 1
-7 -5 -3 -1 1 3 5 7
0 1 0 1 0 1 0 1Bit b3
8-PAM
Polarization Polar coding Performance
Example: 8-PAM as 3 bit channels
I PAM signals selected by three bits (b1, b2, b3)
I Three layers of binary channels created
I Each layer encoded independently
I Layers decoded in the order b3, b2, b1
Bit b2 0 1 10
Bit b1 0 1
-7 -5 -3 -1 1 3 5 7
0 1 0 1 0 1 0 1Bit b3
8-PAM
Polarization Polar coding Performance
Example: 8-PAM as 3 bit channels
I PAM signals selected by three bits (b1, b2, b3)
I Three layers of binary channels created
I Each layer encoded independently
I Layers decoded in the order b3, b2, b1
Bit b2 0 1 10
Bit b1 0 1
-7 -5 -3 -1 1 3 5 7
0 1 0 1 0 1 0 1Bit b3
8-PAM
Polarization Polar coding Performance
Example: 8-PAM as 3 bit channels
I PAM signals selected by three bits (b1, b2, b3)
I Three layers of binary channels created
I Each layer encoded independently
I Layers decoded in the order b3, b2, b1
Bit b2 0 1 10
Bit b1 0 1
-7 -5 -3 -1 1 3 5 7
0 1 0 1 0 1 0 1Bit b3
8-PAM
Polarization Polar coding Performance
Multi-layering jump-starts polarization
0 5 10 15 200
0.5
1
1.5
2
2.5
3
3.5
SNR (dB)
Cap
acity
(bi
ts)
Layer 1 capacityLayer 2 capacityLayer 3 capacitySum of three layersShannon limit
Polarization Polar coding Performance
4-QAM, Rate 1/2
−1 0 1 2 3 4 5 610
−6
10−5
10−4
10−3
10−2
10−1
100
EbNo (dB)
FE
R
Polar(512,256) 4−QAMPolar(1024,512) 4−QAMCTC(480,240) 4−QAMCTC(960,480) 4−QAM
Turbo
Polar
Polarization Polar coding Performance
16-QAM, Rate 3/4
3 3.5 4 4.5 5 5.5 6 6.5 7 7.5 8 8.510
−6
10−5
10−4
10−3
10−2
10−1
100
EbNo (dB)
FE
R
Polar(512,384) 16−QAMCTC(192,144) 16−QAMCTC(384,288) 16−QAMCTC(576,432) 16−QAM
Polarization Polar coding Performance
64-QAM, Rate 5/6
7.5 8 8.5 9 9.5 10 10.5 11 11.5 12 12.5 1310
−5
10−4
10−3
10−2
10−1
100
EbNo (dB)
FE
R
Polar(768,640) 64−QAMPolar(384,320) 64−QAMCTC(576,480) 64−QAM
Polarization Polar coding Performance
Complexity comparison: 64-QAM, Rate 5/6
Average decoding time in milliseconds per codeword (ms/cw)
Eb/N0 CTC(576,432) Polar(768,640) Polar(384,320)
10 dB 6.23 0.92 0.4811 dB 1.83 1.01 0.53
Both decoders implemented as MATLAB mex functions. Polar decoder is a successive
cancellation decoder. CTC decoder is a public domain decoder (CML). Profiling done
by MATLAB Profiler. Iteration limit for CTC decoder was 10; average no of iterations
was 10 at 10 dB and 3.3 at 11 dB. CTC decoder used a linear approximation to
log-MAP while polar decoder used exact log-MAP.
Polarization Polar coding Performance
Complexity comparison: 64-QAM, Rate 5/6
Average decoding time in milliseconds per codeword (ms/cw)
Eb/N0 CTC(576,432) Polar(768,640) Polar(384,320)
10 dB 6.23 0.92 0.4811 dB 1.83 1.01 0.53
Polar codes show a complexity advantage against CTC codes.
Both decoders implemented as MATLAB mex functions. Polar decoder is a successive
cancellation decoder. CTC decoder is a public domain decoder (CML). Profiling done
by MATLAB Profiler. Iteration limit for CTC decoder was 10; average no of iterations
was 10 at 10 dB and 3.3 at 11 dB. CTC decoder used a linear approximation to
log-MAP while polar decoder used exact log-MAP.
Polarization Polar coding Performance
Performance improvement for polar codes
I Concatenation to improve minimum distance
I List decoding to improve SC decoder performance
Polarization Polar coding Performance
Performance improvement for polar codes
I Concatenation to improve minimum distance
I List decoding to improve SC decoder performance
Polarization Polar coding Performance
Concatenation
Method Ref
Block turbo coding with polar constituents AKMOP (2009)Generalized concatenated coding with polar inner AM (2009)Reed-Solomon outer, polar inner BJE (2010)Polar outer, block inner SH (2010)Polar outer, LDPC inner EP (ISIT’2011)
AKMOP: A., Kim, Markarian, Ozgur, PoyrazGCC: A., MarkarianBJE: Bakshi, Jaggi, and EffrosSH: Seidl and HuberEP: Eslami and Pishro-Nik
Polarization Polar coding Performance
Tal-Vardy list decoder for polar codes
I First produce L candidate decisions
I Pick the most likely word from the list
I Complexity O(LN logN)
Polarization Polar coding Performance
Tal-Vardy list decoder for polar codes
I First produce L candidate decisions
I Pick the most likely word from the list
I Complexity O(LN logN)
Polarization Polar coding Performance
Tal-Vardy list decoder for polar codes
I First produce L candidate decisions
I Pick the most likely word from the list
I Complexity O(LN logN)
Polarization Polar coding Performance
Tal-Vardy list decoder performanceLength n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polarization Polar coding Performance
Tal-Vardy list decoder performanceLength n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polarization Polar coding Performance
Tal-Vardy list decoder performanceLength n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polarization Polar coding Performance
Tal-Vardy list decoder performanceLength n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polarization Polar coding Performance
Tal-Vardy list decoder performance
Length n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polarization Polar coding Performance
Tal-Vardy list decoder performance
Length n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polarization Polar coding Performance
Tal-Vardy list decoder performance
Length n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
List-of-L performance quickly approaches ML performance!
Polarization Polar coding Performance
Tal-Vardy list decoder with CRC
I Same decoder as before but data contains a built-in CRC
I Selection made by CRC and relative likelihood
Polarization Polar coding Performance
Tal-Vardy list decoder with CRC
I Same decoder as before but data contains a built-in CRC
I Selection made by CRC and relative likelihood
Polarization Polar coding Performance
Tal-Vardy list decoder with CRC
Length n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polarization Polar coding Performance
Tal-Vardy list decoder with CRC
Length n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polarization Polar coding Performance
Tal-Vardy list decoder with CRC
Length n = 2048, rate R = 0.5, BPSK-AWGN channel, list-size L.
Polar codes (+CRC) achieve state-of-the-art performance!
Polarization Polar coding Performance
Hardware implementation of polar codes
I AdvantagesI Regular structure simplifies resource reuseI Lack of randomness helps avoid memory conflicts
I DisadvantagesI High latency: O(N)I Throughput bottleneck: 1/2 bits per clock-period
References: A. (2007, 2010), Leroux, Tal, Vardy, Gross (2010),Leroux, Sarkis, and Gross (2011), Pamuk and A. (2011).
Polarization Polar coding Performance
Hardware implementation of polar codes
I AdvantagesI Regular structure simplifies resource reuseI Lack of randomness helps avoid memory conflicts
I DisadvantagesI High latency: O(N)I Throughput bottleneck: 1/2 bits per clock-period
References: A. (2007, 2010), Leroux, Tal, Vardy, Gross (2010),Leroux, Sarkis, and Gross (2011), Pamuk and A. (2011).
Polarization Polar coding Performance
Hardware implementation of polar codes
I AdvantagesI Regular structure simplifies resource reuseI Lack of randomness helps avoid memory conflicts
I DisadvantagesI High latency: O(N)I Throughput bottleneck: 1/2 bits per clock-period
References: A. (2007, 2010), Leroux, Tal, Vardy, Gross (2010),Leroux, Sarkis, and Gross (2011), Pamuk and A. (2011).
Polarization Polar coding Performance
Hardware implementation of polar codes
I AdvantagesI Regular structure simplifies resource reuseI Lack of randomness helps avoid memory conflicts
I DisadvantagesI High latency: O(N)I Throughput bottleneck: 1/2 bits per clock-period
References: A. (2007, 2010), Leroux, Tal, Vardy, Gross (2010),Leroux, Sarkis, and Gross (2011), Pamuk and A. (2011).
Polarization Polar coding Performance
Hardware implementation of polar codes
I AdvantagesI Regular structure simplifies resource reuseI Lack of randomness helps avoid memory conflicts
I DisadvantagesI High latency: O(N)I Throughput bottleneck: 1/2 bits per clock-period
References: A. (2007, 2010), Leroux, Tal, Vardy, Gross (2010),Leroux, Sarkis, and Gross (2011), Pamuk and A. (2011).
Polarization Polar coding Performance
Hardware implementation of polar codes
I AdvantagesI Regular structure simplifies resource reuseI Lack of randomness helps avoid memory conflicts
I DisadvantagesI High latency: O(N)I Throughput bottleneck: 1/2 bits per clock-period
References: A. (2007, 2010), Leroux, Tal, Vardy, Gross (2010),Leroux, Sarkis, and Gross (2011), Pamuk and A. (2011).
Polarization Polar coding Performance
Summary
I Polarization is a commonplace phenomenon – almostunavoidable
I Polar codes are low-complexity methods designed to exploitpolarization for achieving Shannon limits
I Polar codes with some help from other methods performcompetitively with the state-of-the-art codes in terms ofcomplexity and performance
Polarization Polar coding Performance
Summary
I Polarization is a commonplace phenomenon – almostunavoidable
I Polar codes are low-complexity methods designed to exploitpolarization for achieving Shannon limits
I Polar codes with some help from other methods performcompetitively with the state-of-the-art codes in terms ofcomplexity and performance
Polarization Polar coding Performance
Summary
I Polarization is a commonplace phenomenon – almostunavoidable
I Polar codes are low-complexity methods designed to exploitpolarization for achieving Shannon limits
I Polar codes with some help from other methods performcompetitively with the state-of-the-art codes in terms ofcomplexity and performance
Polarization Polar coding Performance
Acknowledgements
Research sponsors
Special thanks to Emre Telatar, Alex Vardy, Ido Tal, and WarrenGross for sharing ideas, software and slides, and to MatthewValenti for Coded Modulation Library.
Polarization Polar coding Performance
Thank you!