error control coding - electrical engineeringagullive/introduction.pdf · 2008 error control coding...
TRANSCRIPT
![Page 1: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/1.jpg)
Error Control Coding
Aaron Gulliver
Dept. of Electrical and Computer Engineering
University of Victoria
![Page 2: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/2.jpg)
Topics
• Introduction – The Channel Coding Problem
• Linear Block Codes
• Cyclic Codes
• BCH and Reed-Solomon Codes
• Convolutional Codes
22008 Error Control Coding
![Page 3: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/3.jpg)
Errors in Information TransmissionDigital Communications:Transporting information from one place to another using a sequence of symbols, e.g. bits.
Received bits: a corrupted version of the transmitted bits
Transmitted bits
…. 0110010101 …
…. 0100010101 …
Noise & interference:The received sequence may differ from the transmitted one.
32008 Error Control Coding
![Page 4: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/4.jpg)
Magnetic recording Track
Sector
010101011110010101001 Some of the bits may change during the transmission from the disk to the disk drive
Errors in Information Transmission
42008 Error Control Coding
![Page 5: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/5.jpg)
Deep Space Communications
52008 Error Control Coding
![Page 6: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/6.jpg)
ISBN Codes
62008 Error Control Coding
![Page 7: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/7.jpg)
Bar Codes
72008 Error Control Coding
![Page 8: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/8.jpg)
Errors
82008 Error Control Coding
![Page 9: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/9.jpg)
Digital Communication System
92008 Error Control Coding
![Page 10: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/10.jpg)
102008 Error Control Coding
![Page 11: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/11.jpg)
Performance in AWGN
• Consider an AWGN channel with SNR Eb/N0
• BERBPSK =
• BERBFSK =
2008 Error Control Coding 11
0
2 bEQ
N
0/21
2
bE Ne
2 /21( )
2
y
x
Q x e dy
![Page 12: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/12.jpg)
Performance of BPSK and BFSK
122008 Error Control Coding
![Page 13: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/13.jpg)
132008 Error Control Coding
![Page 14: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/14.jpg)
Communications System Design Goals
• Maximize bit rate - Rb
• Minimize the probability of error (BER) - Pb
• Minimize required SNR - Eb/N0
• Minimize required bandwidth - W
• Maximize system utilization - Capacity
• Minimize system complexity and cost - $
142008 Error Control Coding
![Page 15: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/15.jpg)
System Constraints
• Minimum bandwidth based on the modulation used
• Channel capacity
• Government regulations
• Technological limitations
• Other requirements (i.e. cost and complexity)
152008 Error Control Coding
![Page 16: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/16.jpg)
Claude Shannon (1916-2001)
162008 Error Control Coding
![Page 17: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/17.jpg)
A Mathematical Theory of Communications, BSTJ, July 1948
``The fundamental problem of communication is that of reproducing at one point exactly or approximately a message selected at another point. …
If the channel is noisy it is not in general possible to reconstruct the original message or the transmitted signal with certainty by any operation on the received signal.’’
172008 Error Control Coding
![Page 18: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/18.jpg)
Channel Capacity
18
• An important question for a communication channel is the maximum rate at which it can transfer information.
• The channel capacity C is a theoretical maximum rate below which information can be transmittederror free over the channel.
• The channel capacity C is given by
C = W log(1+P/N0W) b/s
where W is the bandwidth, P is the signal power and N0W is the noise power.
2008 Error Control Coding
![Page 19: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/19.jpg)
Shannon’s Noisy Channel Coding Theorem
With every channel we can associate a``channel capacity’’ C. There exist error controlcodes such that information can betransmitted across the channel at rates lessthan C with arbitrarily low bit error rate.
• There are only two factors that determine the capacity of a channel:
– Bandwidth (W)
– Signal-to-Noise Ratio (SNR) Eb/N0
192008 Error Control Coding
![Page 20: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/20.jpg)
Channel Capacity
Let
20
2
0
2
0
log 1
log 1
b b
b b
PC W
N W
E PT P E R
E RC W
N W
2
0
/
0
log 1
2 1
/
b
C W
b
EC C
W N W
E
N C W
bR C
2008 Error Control Coding
![Page 21: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/21.jpg)
Bandwidth Efficiency Plane
212008 Error Control Coding
![Page 22: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/22.jpg)
Digital Communication Model with Coding
222008 Error Control Coding
![Page 23: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/23.jpg)
Power Efficiency
232008 Error Control Coding
![Page 24: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/24.jpg)
Performance of Turbo Codes
• Comparison:
– Rate R=1/2 Codes
– K=5 turbo code
– K=14 convolutional code
Gain of almost 2 dB!
Theoretical Limit!
2008 24Error Control Coding
![Page 25: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/25.jpg)
The Turbo Principle
• Turbo codes are so named because the decoder uses feedback, like a turbo-charged engine.
2008 25Error Control Coding
![Page 26: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/26.jpg)
Performance as a Function of Number of Iterations
• K=5
• R=1/2
• n=65,536
0.5 1 1.5 210
-7
10-6
10-5
10-4
10-3
10-2
10-1
100
Eb/N
o in dB
BE
R
1 iteration
2 iterations
3 iterations6 iterations
10 iterations
18 iterations
2008 26Error Control Coding
![Page 27: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/27.jpg)
Performance of Various Codes
272008 Error Control Coding
![Page 28: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/28.jpg)
Code Imperfectness Relative to the Sphere-Packing Bound
282008 Error Control Coding
![Page 29: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/29.jpg)
Randomly Interleaved SPC Product Codes with BER = 10-5
292008 Error Control Coding
![Page 30: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/30.jpg)
ASCII Character Set
2008 Error Control Coding 30
![Page 31: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/31.jpg)
SPC Code – Example 1
• ASCII symbols
E = 1000101 c = 10001011
G = 1000111 c = 10001110
• Received word
r = 10001010
312008 Error Control Coding
![Page 32: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/32.jpg)
Triple Repetition Code – Decoding
Received Word Codeword
111
111
111
111
000
000
000
000
001
010
100
000
001
010
100
0000 0 0
0 0 1
0 1 0
1 0 0
1 1 1
1 1 0
1 0 1
0 1 1
Error Pattern
322008 Error Control Coding
![Page 33: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/33.jpg)
Triple Repetition Code1 2 3( , , ) ( , , )c c c m m m
Encoder: Repeat
each bit three times
),,( 321 rrr
( )m
BSC
codeword
Decoder: Take the
majority vote
Corrupted codeword
m̂
332008 Error Control Coding
![Page 34: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/34.jpg)
Triple Repetition Code (Cont.)
Decoder
Encoder
BSC
codeword
m = (0) (0,0,0)
(1,0,0)
Corrupted codeword
(0)
Successful
decoding!
Decoding: majority voting
342008 Error Control Coding
![Page 35: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/35.jpg)
Information bits Received bits
BSC
10010…10101… 10110…00101…
A communication channel can be modeled as a Binary Symmetric Channel (BSC)
Each bit is flipped with probability p
Transmission Errors
SenderReceiver
352008 Error Control Coding
![Page 36: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/36.jpg)
Binary Symmetric Channel• Transmitted symbols are binary
• Errors affect 0s and 1s with equal probability (symmetric)
• Errors occur randomly and are independent from bit to bit (memoryless)
INPUT OUTPUT
0 0
1 1
1-p
1-p
p
p
p is the probability of bit error - Crossover probability
362008 Error Control Coding
![Page 37: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/37.jpg)
Binary Symmetric Channel
• If n symbols are transmitted, the probability of an m error pattern is
• The probability of exactly m errors is
• The probability of m or more errors is
mnm pp
1
n
m
mnm pp
1
1n
n ii n
i
i m
p p
372008 Error Control Coding
![Page 38: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/38.jpg)
Example
• The BSC bit error probability is p <• majority vote or nearest neighbor decoding
000, 001, 010, 100 000 111, 110, 101, 011 111
• the probability of a decoding error is
• Example: If p = 0.01, then Pe(C) = 0.000298 and only one word in 3555 will be in error after decoding.
2
1
pppppp 3232 23)1(3
382008 Error Control Coding
![Page 39: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/39.jpg)
Single Parity Check Code Example• Consider the 211 binary words of length 11 as
datawords• Let the probability of error be 10-8
• Bits are transmitted at a rate of 107 bits per second• The probability that a word is transmitted incorrectly is
approximately
• Therefore words per second are transmitted incorrectly.
• One wrong word is transmitted every 10 seconds, 360 erroneous words every hour and 8640 words every day without being detected!
8
10
10
11111 pp
1.01110
10
117
8
392008 Error Control Coding
![Page 40: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/40.jpg)
SPC Code Example (Cont.)
• Let one parity bit be added• Any single error can be detected• The probability of at least 2 errors is
• Therefore approximately words per second are transmitted with an undetectableerror
• An undetected error occurs only every 2000 days (2000 109/(5.5 86400))
16
21012
2
1112
10
66111211 ppppp
9
1210
10
66 105.57
16
402008 Error Control Coding
![Page 41: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/41.jpg)
Performance of BPSK and BFSK
412008 Error Control Coding
![Page 42: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/42.jpg)
Coded Performance• BPSK modulation in an AWGN channel
– Eb/N0 = 10 dB
– Demodulated BER = 3.910-6
• Using the (15,11) Hamming code– Code rate is 11/15 = 0.73
– Received Eb/N0 = 8.65 dB (loss 1.35 dB)
– Demodulated BER = 5.410-5
• After decoding (single error correcting code)– BER = 5.610-8
– Coding gain 11.5 – 10 = 1.5 dB
2008 Error Control Coding 42
![Page 43: Error Control Coding - Electrical engineeringagullive/Introduction.pdf · 2008 Error Control Coding 17. Channel Capacity 18 •An important question for a communication channel is](https://reader030.vdocument.in/reader030/viewer/2022021717/5b3c2ab87f8b9a560a8d3a29/html5/thumbnails/43.jpg)
Coding Gain with the (15,11) Hamming Code
432008 Error Control Coding