Polar Codes -- A New Paradigm for Coding
R. Urbanke, EPFL
Physics of Algorithms, Santa Fe, September 2nd, 2009
Thanks to Emre Telatar and Satish Korada.(for many borrowed figures)
http://panorama.epfl.chSunday, September 13, 2009
Sunday, September 13, 2009
Coding
Sunday, September 13, 2009
Coding
Sunday, September 13, 2009
Coding
codeC={000, 010, 101, 111}
n ... blocklength
Sunday, September 13, 2009
Important Parameters
Sunday, September 13, 2009
Important Parameters
rate, error probability, encoding complexity,decoding complexity, blocklength
Sunday, September 13, 2009
Linear Codes
Sunday, September 13, 2009
Linear Codes
generator matrix
Sunday, September 13, 2009
Linear Codes
generator matrix parity-check matrix
Sunday, September 13, 2009
Linear Codes
generator matrix parity-check matrix
Sunday, September 13, 2009
Bitwise MAP Decoding
Sunday, September 13, 2009
Bitwise MAP Decoding
[LDPC -- Gallager ‘60]
Sunday, September 13, 2009
Polar Codes: Summary
Sunday, September 13, 2009
Polar Codes: Summary
Erdal Arikan, ISIT 2007
Sunday, September 13, 2009
Polar Codes: Summary
Erdal Arikan, ISIT 2007
very general phenomenon
Sunday, September 13, 2009
Polar Codes: Summary
Erdal Arikan, ISIT 2007
very general phenomenon
information theoretic view whycodes work
Sunday, September 13, 2009
Polar Codes: Summary
Erdal Arikan, ISIT 2007
very general phenomenon
information theoretic view whycodes work
first “low complexity” scheme which provablyachieves the capacity for a fairly wide array of
channels
Sunday, September 13, 2009
Polar Codes: Summary
Erdal Arikan, ISIT 2007
very general phenomenon
information theoretic view whycodes work
first “low complexity” scheme which provablyachieves the capacity for a fairly wide array of
channels
many possible variations on the theme
Sunday, September 13, 2009
Polar Codes: Summary
Erdal Arikan, ISIT 2007
very general phenomenon
information theoretic view whycodes work
first “low complexity” scheme which provablyachieves the capacity for a fairly wide array of
channels
codes not only good for channel coding;work equally well for source coding and more
complicated scenarios
many possible variations on the theme
Sunday, September 13, 2009
References
Sunday, September 13, 2009
References
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Codes from Kronecker Product of G2
Sunday, September 13, 2009
Reed-Muller Codes
Sunday, September 13, 2009
Reed-Muller Codes
choose rows of largest weight
Sunday, September 13, 2009
Definition of Channels
Polar Codes
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Polar Codes
W -- BMS channel
Sunday, September 13, 2009
Definition of Channels
Channel Polarization
Sunday, September 13, 2009
Definition of Channels
Channel Polarization
Sunday, September 13, 2009
Definition of Channels
Channel Polarization
Sunday, September 13, 2009
Definition of Channels
Channel Polarization
Sunday, September 13, 2009
Channel Polarization
Definition of Channels
Sunday, September 13, 2009
Channel Polarization
Definition of Channels
N
Sunday, September 13, 2009
Channel Polarization
Definition of Channels
N
Sunday, September 13, 2009
Channel Polarization
Definition of Channels
N
Sunday, September 13, 2009
Channel Polarization
Definition of Channels
N
Sunday, September 13, 2009
Channel Polarization
Definition of Channels
N
Sunday, September 13, 2009
Channel Polarization
Definition of Channels
N
Sunday, September 13, 2009
Channel Polarization
Definition of Channels
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
Successive Decoding
Sunday, September 13, 2009
Definition of Channels
More on Polarization
Sunday, September 13, 2009
Definition of Channels
More on Polarization
Sunday, September 13, 2009
Definition of Channels
More on Polarization
Sunday, September 13, 2009
Definition of Channels
More on Polarization
Sunday, September 13, 2009
Definition of Channels
More on Polarization
Sunday, September 13, 2009
Definition of Channels
More on Polarization
N
Sunday, September 13, 2009
Definition of Channels
More on Polarization
N
Sunday, September 13, 2009
Definition of Channels
More on Polarization
Sunday, September 13, 2009
Definition of Channels
N
More on Polarization
Sunday, September 13, 2009
Definition of Channels
N
More on Polarization
Sunday, September 13, 2009
Definition of Channels
N
More on Polarization
Sunday, September 13, 2009
Definition of Channels
N
More on Polarization
Sunday, September 13, 2009
Equivalent “Random” Channel
Sunday, September 13, 2009
Equivalent “Random” Channel
Set B1, B2, ... to be i.i.d. {+, -} valued, uniformly distributed random variables
Sunday, September 13, 2009
Equivalent “Random” Channel
Define In=I(WB1, B2, ..., Bn)
Set B1, B2, ... to be i.i.d. {+, -} valued, uniformly distributed random variables
Sunday, September 13, 2009
Equivalent “Random” Channel
Define In=I(WB1, B2, ..., Bn)
Set B1, B2, ... to be i.i.d. {+, -} valued, uniformly distributed random variables
Study the distribution of In
Sunday, September 13, 2009
Properties of In
Sunday, September 13, 2009
Properties of In
I0=I(W) is a constant
Sunday, September 13, 2009
Properties of In
In ∈ [0, 1]; so In is bounded
I0=I(W) is a constant
Sunday, September 13, 2009
Properties of In
In ∈ [0, 1]; so In is bounded
I0=I(W) is a constant
Conditional on B1, B2, ..., Bn, and withP= WB1, B2, ..., Bn, In+1 can only take on thetwo values I(P+) and I(P-)
Sunday, September 13, 2009
Properties of In
In ∈ [0, 1]; so In is bounded
I0=I(W) is a constant
Conditional on B1, B2, ..., Bn, and withP= WB1, B2, ..., Bn, In+1 can only take on thetwo values I(P+) and I(P-)
Further, E[In+1 | B1, B2, ..., Bn]=(I(P+)+ I(P-))/2=I(P), so {In} is a (bounded) martingale
Sunday, September 13, 2009
Properties of In
Sunday, September 13, 2009
a bounded martingale converges almost surely
Properties of In
Sunday, September 13, 2009
I∞ = limn→∞ In exists almost surely; E[I∞]=I0=I(W)
a bounded martingale converges almost surely
Properties of In
Sunday, September 13, 2009
I∞ = limn→∞ In exists almost surely; E[I∞]=I0=I(W)
a bounded martingale converges almost surely
Pr{|In+1-In|≤ε}→1; but |In+1-In|=(I(P+)- I(P-))/2
Properties of In
Sunday, September 13, 2009
I∞ = limn→∞ In exists almost surely; E[I∞]=I0=I(W)
a bounded martingale converges almost surely
Pr{|In+1-In|≤ε}→1; but |In+1-In|=(I(P+)- I(P-))/2
from extremes of information combining we knowthat (I(P+)-I(P-))/2 ≤ε implies that I(P)∉(δ, 1-δ)
Properties of In
(I(P+)- I(P-))/2
I(P)
Sunday, September 13, 2009
I∞ = limn→∞ In exists almost surely; E[I∞]=I0=I(W)
a bounded martingale converges almost surely
Pr{|In+1-In|≤ε}→1; but |In+1-In|=(I(P+)- I(P-))/2
from extremes of information combining we knowthat (I(P+)-I(P-))/2 ≤ε implies that I(P)∉(δ, 1-δ)
Properties of In
we conclude that I∞ takes values only in {0, 1}
(I(P+)- I(P-))/2
I(P)
Sunday, September 13, 2009
Summary of Known Results
Sunday, September 13, 2009
Summary of Known Results
achieve capacity on memoryless channels Arikan 2007
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
achieve capacity on memoryless channels Arikan 2007
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
Korada, Sasoglu, and U. 2009
achieve capacity on memoryless channels Arikan 2007
Sunday, September 13, 2009
Definition of Channels
Polar Codes Based on Larger Matrices
Sunday, September 13, 2009
Definition of Channels
Polar Codes Based on Larger Matrices
Sunday, September 13, 2009
Definition of Channels
Characterization of Exponent
Sunday, September 13, 2009
Definition of Channels
Exponent: Example
Sunday, September 13, 2009
Definition of Channels
Exponent: Example
Sunday, September 13, 2009
Definition of Channels
Exponent: Example
Sunday, September 13, 2009
Definition of Channels
Exponent: Example
Sunday, September 13, 2009
Definition of Channels
Exponent: Example
Sunday, September 13, 2009
Definition of Channels
Exponent: Example
Sunday, September 13, 2009
Definition of Channels
Results
Sunday, September 13, 2009
Definition of Channels
Results
Sunday, September 13, 2009
Definition of Channels
Results
Sunday, September 13, 2009
Definition of Channels
Results
Sunday, September 13, 2009
Definition of Channels
Results
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
Korada, Sasoglu, and U. 2009
achieve capacity on memoryless channels Arikan 2007
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
Korada, Sasoglu, and U. 2009
optimal for lossy source coding, Wyner-Ziv, Gelfand-Pinsker, ...
Korada and U. 2009
achieve capacity on memoryless channels Arikan 2007
Sunday, September 13, 2009
Source Coding
Definition of Channels
Sunday, September 13, 2009
Source Coding
Definition of Channels
Sunday, September 13, 2009
Source Coding
Definition of Channels
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
Korada, Sasoglu, and U. 2009
optimal for lossy source coding, Wyner-Ziv, Gelfand-Pinsker, ...
Korada and U. 2009
achieve capacity on memoryless channels Arikan 2007
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
Korada, Sasoglu, and U. 2009
optimal for lossy source coding, Wyner-Ziv, Gelfand-Pinsker, ...
Korada and U. 2009
Mori and Tanaka 2009efficient construction
achieve capacity on memoryless channels Arikan 2007
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
Korada, Sasoglu, and U. 2009
optimal for lossy source coding, Wyner-Ziv, Gelfand-Pinsker, ...
Korada and U. 2009
suboptimal for compound coding Hassani, Korada and U. 2009
Mori and Tanaka 2009efficient construction
achieve capacity on memoryless channels Arikan 2007
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
Korada, Sasoglu, and U. 2009
optimal for lossy source coding, Wyner-Ziv, Gelfand-Pinsker, ...
Korada and U. 2009
suboptimal for compound coding Hassani, Korada and U. 2009
Mori and Tanaka 2009efficient construction
achieve capacity on memoryless channels Arikan 2007
non-binary version and asym. channels Arikan, Sasoglu, and Telatar 2009
Sunday, September 13, 2009
Summary of Known Results
Arikan and Telatar 2008
Korada, Sasoglu, and U. 2009
optimal for lossy source coding, Wyner-Ziv, Gelfand-Pinsker, ...
Korada and U. 2009
suboptimal for compound coding Hassani, Korada and U. 2009
Mori and Tanaka 2009efficient construction
achieve capacity on memoryless channels Arikan 2007
non-binary version and asym. channels Arikan, Sasoglu, and Telatar 2009
scaling
Sunday, September 13, 2009
Summary
+ many applications
+ completely new paradigm of coding
+ provably achieves capacity
+ low complexity
- currently only competitive for VERY large N
Sunday, September 13, 2009
Wyner-Ziv and Gelfand-Pinsker
Sunday, September 13, 2009