Download - Fountain Codes
Fountain CodesD.J.C MacKay
IEE Proceedings Communications, Vol. 152, No. 6, December 2005
2
Introduction Fountain Codes Intermission LT Codes Raptor Codes
Outline
3
Erasure Channel:◦ Files are transmitted in multiple small packets.◦ Each packet is either received without error or
loss.◦ Such as the Internet.
How to deal with packet loss?◦ Some simple retransmission protocols:
ACKs: for missing packets. ACKs: for received packets.
◦ Erasure Correcting Codes.
Introduction
4
Why Erasure Correcting Codes? Retransmission are wasteful when erasure is
serious: ACKs: for missing packets.
ACKs would be enormous.
ACKs: for received packets. Would lead to multiple copies.
Broadcast Channel with erasure:
Introduction
server
ADC
FE
5
Erasure Correcting Codes: Block Code, such as (N, K) Reed-Solomon Code:
Any K of the N transmitted symbols are received, then the
original K source symbols can be recovered.
High Complexity: O( K(N-K) log2N)
Estimate the erasure probability f first, then choose the
code rate R = K/N before transmission.
Ex. If loss-rate = 50%, then set code rate R = 1/(1-50%) =
1/2 = K/N.
( N = 2K )
Introduction
loss-rate = 50%
6
Erasure Correcting Codes: Block Code, such as (N, K) Reed-Solomon Code:
If f is larger than expected (decoder receives fewer than K
symbols):
Ex. We thought loss-rate is 50%, and set the code rate R =
1/2 ( N = 2K ); however, the actual loss-rate = 66.7%, the
proper code rate R should be lower: R = 1/3 ( N = 3K )
We would like a simple way to extend the code on the fly to
create a lower-rate (N’, K) code.
Introduction
loss-rate = 66.7%
No way!
Fountain Codes are rateless: The number of encoded packets generated can be
determined on the fly.
Fountain Codes
7
Fountain Codes are rateless: The number of encoded packets generated can be
determined on the fly.
Fountain Codes can also have fantastically small encoding and decoding complexities. Depends on the careful choice of Degree Distribution.
Fountain Codes
9
Balls–and–Bins Problem:◦ Imagine that we throw N balls independently at
random into K bins, what probability of one bin have no balls in it?
Intermission
…K bins
…N throws
10
Balls–and–Bins Problem:◦ After N balls have been thrown, what probability
of one bin have no ball in it? The probability that one particular bin is empty after
N balls have been thrown:
Intermission
…K
bins
11
Balls–and–Bins Problem:◦ After N balls have been thrown, what probability of
one bin have no ball in it? The probability that one particular bin is empty after
N balls have been thrown:
The expected number of empty bins: δ =
,which roughly implies: the probability of all bins have a ball is (1- δ) only if:
Intermission …K
bins
12
Luby Transform (LT) Codes:◦ Encoding process:
For the ith encoded packet, select degree di by carefully chosen Degree Distribution (Robust Soliton Distribution).
Choose di source data. Perform XOR on chosen data.
◦ Decoding process: Decode degree-one encoded packets. Remove degree-one edges iteratively.
LT Codes
Degree 1 2 3 … k
probability μ1 μ2 μ3 … μk
…
x1 x2 x3 x4 x5
y1 y2 y3 y4 y5
x1
x3
x2
x2
x5
x4
x6
x3
x5
x6
13
Designing the Degree Distribution:◦ A few encoded packets must have high degree.
To ensure that every source data are connected to encoded packets.
◦ Many encoded packets must have low degree. So that decoding process can get started, and keep
going. Also the total number of XOR operations involved in
the encoding and decoding is kept small.
LT Codes
x1 x2 x3 x4 x5
y1 y2 y3 y4 y5
14
Some properties of Degree Distribution:◦ The complexity (both encoding and decoding) are
scaled linearly with the number of edges in the graph.
◦ Key factor: The average degree of the encoded packets.
How small (number of edges) can this be?◦ Recall: Balls–and–Bins Problem.
Balls: linked edges. Bins: source data.
LT Codes
x1 x2 x3 x4 x5
y1 y2 y3 y4 y5
How small number of edges can assure that every source data must have at least one edge on it? (all bins have a ball)
Some properties of encoder:◦ Encoder throws edges into source data at
random. The number of edges must be at least of order : K
lnK. Balls–and–Bins Problem:
The expected number of empty bins: δ =
,which roughly implies: the probability of all bins have a
ball is (1- δ) only if:
LT Codes
15
For decoder:◦ If decoder received optimal K encoded packets, the
average degree of each encoded packet must be at least: lnK The number of edges must be at least of order: K lnK. The complexity (both encoding and decoding) of an LT
code will definitely be at least: K lnK Luby showed that this bound on complexity
can indeed be achieved by a careful choice of Degree Distribution.
LT Codes
16
Ideally, to avoid redundancy:◦ We would like just one check node has degree
one at each iteration. Ideal Soliton Distribution:
The expected average degree under this distribution is roughly: lnK
LT Codes
17
In practice, this distribution works poorly:◦ Fluctuations around the expected behavior:
Sometimes in the decoding process there will be no degree-one check node.
◦ A few source data will receive no connections at all.
Some small modification fixes these problems.◦ Robust Soliton Distribution:
More degree-one check node. A bit more high-degree check node.
LT Codes
18
19
Robust Soliton Distribution: Two extra parameters: c and δ The expected number of degree-one check node
(through out decoding process) is about:
δ: a bound on the decoding failure probability. Decoding fails to run to completion after K’ of encoded
packets have been received. c: a free parameters smaller than 1.
Luby’s key result.
LT Codes
20
Luby defines a positive function:
, then adds the Ideal Soliton Distribution ρ to τ and normalize to obtain the Robust Soliton Distribution μ:
LT Codes
, where ※ Receiver once receives K' = KZ encoded packets
ensures that the decoding can run to completion with probability at least 1 - δ.
21
LT Codes
τ( k/S )
※ High-degree ensures every source data is likely to be connected to a check.
※ Small-degree of τ ensures the decoding process gets started.
22
LT Codes※ Histograms of the number of encoded packets N required in order to recover source data K = 10,000
23
LT Codes ※ Practical performance of LT codes
- Three experimental decodings are shown.
※ All codes created with c = 0.03, δ = 0.5 (S= 30, K/S = 337, Z = 1.03), and K = 10,000
an overhead of 10%
24
Complexity cost:◦ LT Codes: O(K lnK), where K: the number of
original data. Average degree of the encoded packets: lnK Encoding and decoding complexity: lnK per
encoded packet◦ Raptor Codes: Linear time encoding and
decoding. Concatenating a weakened LT Code with an outer
code. Average degree of weakened LT code ≒ 3
Raptor Codes
25
Weakened LT Code:◦ Average degree of encoded packets ≒ 3◦ A fraction of source data will receive no
connections at all. What fraction?
◦ Balls–and–Bins Problem:
Raptor Codes
Also the fraction of empty bins
≒ 5%
26
Shokrollahi’s trick: ※ Encoder:
◦ Find a outer code can correct erasures if the erasure rate is:, then pre-code K source data into:
◦ Transmit this slightly enlarged data using a weaken LT Code. ※ Decoder:
◦ Once slightly more than K encoded packets been received, can recover of the pre-coded packets (roughly K packets).
◦ Then use the outer code to recover all the original data.
Raptor Codes
27
Raptor CodesK = 16
K’ = 20
N = 18
※ Schematic diagram of a Raptor Code
Pre-Coding
Weaken LT
covered = 17
28
Raptor Codes※ The idea of a weakened LT Code.
※ LT codes created with c = 0.03, δ = 0.5 and truncated at degree 8 (thus average degree = 3)
Thank you :)
30
Soliton Distribution Ideal Soliton Distribution:
The average degree is roughly: lnK
)ln( 1
)1(111
)(
1
2
1
Kd
ddd
K
ddd
K
d
K
d
K
d
31
Soliton Distribution Robust Soliton Distribution μ:
The average degree is roughly: lnK
)ln( )ln(1)ln(
)ln(1
1
) (
) () (
1
1
1
2
KOSK
SKS
d
ddd
ddddd
d
RK
d
K
d
d
d
d
, where