EE4723 : Digital Communications II
Lecture 25:
Channel Encoding: Block Codes
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 1
Linear block codes
� Linear block code (n,k)
– A set with cardinality is called a linear block code if, and only if, it is a subspace of the vector space .nV
nVC ⊂ k2
VCV ⊂→
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 2
• Members of C are called code-words.• The all-zero codeword is a codeword.• Any linear combination of code-words is a codeword.
nk VCV ⊂→
Linear block codes – cont’d
nVkV
C
mapping
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 3
C
Bases of C
� The subset chosen for the code should include as many as elements to reduce the redundancy but they should be as apart as possible to maintain good error performances
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 4
Figure 6.10: Linear block-code structure
Linear block codes – cont’d
� The information bit stream is chopped into blocks of k bits.
� Each block is encoded to a larger block of n bits.
� The coded bits are modulated and sent over channel.
� The reverse procedure is done at the receiver.
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 5
Data blockChannelencoder Codeword
k bits n bits
rate Code
bits Redundant
n
kR
n-k
c =
Generator Matrix� If k is large, a lookup table implementation of the encoder becomes
prohibitive
� Let the set of 2k codewords {U} be described as:
U=m1V1+ m2V2 + …..+ mkVk
� In general, generator matrix can be defined by the following k x n array:
vvvV L
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 6
array:
� Generation of codeword U:
U=mG
=
=
knkk
n
n
k vvv
vvv
vvv
V
V
V
G
L
M
L
L
M
21
22221
11211
2
1
Linear block codes –cont’d
nVkV
C
Bases of C
mapping
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 7
– A matrix G is constructed by taking as its rows the vectors on the basis, .
Bases of C
},,,{ 21 kVVV K
=
=
knkk
n
n
k vvv
vvv
vvv
L
MOM
L
L
M
21
22221
112111
V
V
G
Linear block codes – cont’d
� Encoding in (n,k) block code
mGU =mmmuuu
V
V
⋅= 2
1
),,,(),,,( KK
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 8
– The rows of G, are linearly independent.
kn
k
kn
mmmuuu
mmmuuu
VVV
V
V
⋅++⋅+⋅=
⋅=
2221121
22121
),,,(
),,,(),,,(
KK
MKK
Linear block codes – cont’d
=
=1
0
0
0
1
0
0
0
1
1
1
0
0
1
1
1
0
1
3
2
1
V
V
V
G
1
1
0
0
0
1
1
0
1
1
1
1
1
0
0
0
1
1
1
1
0
0
0
1
1
0
1
0
0
0
1
0
0
0
1
0
1
0
0
1
1
0
0
1
0
0
0
0
1
0
0
0
1
0Message vector Codeword
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 9
� Generate Codeword U4 for the fourth message vector 1 1 0 in Table 6.1
U4 =[ 1 1 0] V=V1+V2+0*V3
= 110100+011010+000000
=101110 ( Codeword for the message vector 110)
1
1
1
1
1
0
0
0
0
1
0
1
1
1
1
1
1
0
101110101
Linear block codes – cont’d
� Systematic block code (n,k)
– A systematic (n,k) linear block code is a mapping a k-dimensional message vector to an n-dimensional code word such that part of the sequence has k message digits and remaining (n-k) are parity digits
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 10
digits and remaining (n-k) are parity digits
– For a systematic code, the first (or last) k elements in the codeword are information bits.
matrix )(
matrixidentity
][
knk
kk
k
k
k
−×=×=
=
P
I
IPG
Systematic Linear Block codes
� A systematic linear code will have a generator matrix
=
=
−
−
−
100
010
001
)(,21
)(,22221
)(,11211
LL
MM
LL
LL
M
M
M
knkkk
kn
kn
k
ppp
ppp
ppp
IPG
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 11
� Combining the previous equations
×…=
−
−
−
100
010
001
].m,m,[mu,....u,u
)(,21
)(,22221
)(,11211
k2121
LL
MM
LL
LL
knkkk
kn
kn
n
ppp
ppp
ppp
Where
ui = m1p1i+ m2p2i+ ….mkpki for i=1,…(n-k)
= mi-n+k for i=(n-k+1)….n
� Thesystematiccodevectorcanbeexpressedas:
Systematic Linear Block codes
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 12
� Thesystematiccodevectorcanbeexpressedas:
44 344 21K
44 344 21K
bitsmessage
k
bitsparity
kn mmmpppU ,,,,,,, 2121 −=
)(,)(,22)(,11
22221212
12121111
knkkknknkn
kk
kk
pmpmpmp
pmpmpmp
pmpmpmp
−−−− +++=+++=+++=
L
L
L
Example
� For (6,3) code example in sec.6.4.3, the codewords can be described as:
[ ]
=M
M
M
100
010
001
101
110
011
,, 321 mmmU
(6.30)
Systematic Linear Block codes
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 13
43421M
43421
3
100101
IP
{ { {654321
321322131 ,,,,,uuuuuu
mmmmmmmmm4342143421321
+++=
(6.30)
(6.31)
Linear block codes – cont’d
� For any linear code we can find a matrix , such that its rows are orthogonal to the rows of G:
nkn ×− )(H
0GH =T
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 14
� H is called the parity check matrix and its rows are linearly independent.
� For systematic linear block codes:
][ Tkn PIH −=
Systematic Linear Block codes
� A systematic linear code will have a generator matrix
=
=
−
−
−
100
010
001
)(,21
)(,22221
)(,11211
LL
MM
LL
LL
M
M
M
knkkk
kn
kn
k
ppp
ppp
ppp
IPG
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 15
� Combining the previous equations
×…=
−
−
−
100
010
001
].m,m,[mu,....u,u
)(,21
)(,22221
)(,11211
k2121
LL
MM
LL
LL
knkkk
kn
kn
n
ppp
ppp
ppp
Where
ui = m1p1i+ m2p2i+ ….mkpki for i=1,…(n-k)
= mi-n+k for i=(n-k+1)….n
� Thesystematiccodevectorcanbeexpressedas:
Systematic Linear Block codes
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 16
� Thesystematiccodevectorcanbeexpressedas:
44 344 21K
44 344 21K
bitsmessage
k
bitsparity
kn mmmpppU ,,,,,,, 2121 −=
)(,)(,22)(,11
22221212
12121111
knkkknknkn
kk
kk
pmpmpmp
pmpmpmp
pmpmpmp
−−−− +++=+++=+++=
L
L
L
Example
� For (6,3) code example in sec.6.4.3, the codewords can be described as:
[ ]
=M
M
M
100
010
001
101
110
011
,, 321 mmmU
(6.30)
Systematic Linear Block codes
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 17
43421M
43421
3
100101
IP
{ { {654321
321322131 ,,,,,uuuuuu
mmmmmmmmm4342143421321
+++=
(6.30)
(6.31)
Linear block codes – cont’d
� For any linear code we can find an matrix , such that its rows are orthogonal to the rows of G:
nkn ×− )(H
0GH =T
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 18
� H is called the parity check matrix and its rows are linearly independent.
� For systematic linear block codes:
][ Tkn PIH −=
Linear block codes – cont’d
FormatChannel encoding
Modulation
Channeldecoding
FormatDemodulation
Detection
Data source
Data sink
U
r
m
m̂
channel
eUr +=
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 19
� Syndrome testing:
– S is syndrome of r, corresponding to the error pattern e.
or vectorpattern error ),....,,(
or vector codeword received ),....,,(
21
21
n
n
eee
rrr
==
e
reUr +=
TT eHrHS ==
Linear block codes – cont’d
� Standard array– For row , find a vector in of
minimum weight which is not already listed in the array.
– Call this pattern and form the row as the
kni −= 2,...,3,2nV
ie
th:i
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 20
corresponding coset
kknknkn
k
k
22222
22222
221
UeUee
UeUee
UUU
⊕⊕
⊕⊕
−−− L
MOLM
L
Lzero
codeword
coset
coset leaders
Linear block codes – cont’d
� Standard array and syndrome table decoding
1. Calculate
2. Find the coset leader, , corresponding to .
3. Calculate and corresponding .
TrHS =
iee =ˆ S
erU ˆˆ += m̂
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 21
– Note that
• If , error is corrected.
• If , undetectable decoding error occurs.
)ˆˆ(ˆˆ e(eUee)UerU ++=++=+=
ee =ˆ
ee ≠ˆ
Linear block codes – cont’d
� Example: Standard array for the (6,3) code
000101110001011111101011101100011000110111000010
000110110010011100101000101111011011110101000001
000111110011011101101001101110011010110100000000
codewords
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 22
010110100101010001
010100100000
100100010000
111100001000
000110110111011010101101101010011100110011000100
LL
M
MMM
Coset leaders
coset
� No column of H can be all zeros, or else an error in the corresponding codeword position would not affect the syndrome and would be undetectable
� All columns of H must be unique. If two columns of H were identical, errors in these two corresponding codeword positions would be indistinguishable
Example: Codeword U = 1 0 1 1 1 0 , and r = 0 0 1 1 1 0 Find S=rHT
Requirements of the parity-check matrix
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 23
Example: Codeword U = 1 0 1 1 1 0 , and r = 0 0 1 1 1 0 Find S=rHT
[ ]
[ ] [ ]00111,11,1
101
110
011
100
010
001
011100
=++=
=
= TrHS
[ ]
[ ]001
000001
=
=
=
T
T
H
eHS
Linear block codes – cont’d
110000100
011000010
101000001
000000000
:computed is of syndrome The
received. is (001110)
ted. transmit(101110)
===
==
r
r
U
TT
Error pattern Syndrome
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 24
111010001
100100000
010010000
001001000
(101110)(100000)(001110)ˆˆ
estimated is vector corrected The
(100000)ˆ
is syndrome this toingcorrespondpattern Error
(100)(001110)
=+=+=
=
===
erU
e
HrHS TT
=100
010
001
][ rrrrrrS
TrHS =� The received signal is multiplied with the parity check matrix:
Decoder implementation
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 25
=
101
110
011][ 654321 rrrrrrS
and
][
][
][
6533
5422
6411
rrrs
rrrs
rrrs
++=
++=
++=
Implementation of decoder
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 26
Figure 6.12: Implementation of the (6,3) decoder
Linear block codes – cont’d
� The Hamming weight of vector U, denoted by w(U), is the number of non-zero elements in U.
� The Hamming distance between two vectors U and V, is the number of elements in which they differ.
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 27
� The minimum distance of a block code is
)()( VUVU, ⊕= wd
)(min),(minmin ii
jiji
wdd UUU ==≠
How it works: 3 dots
Only 3 possible words
Distance Increment = 1
One Excluded State (red)
A B C
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 28
It is really a checksum.
� Single Error Detection
� No error correction
A B C
A C
⇒ Two valid code words (blue)
This is a graphic representation of the “Hamming Distance”
Hamming Distance
� Definition:
– The number of elements that need to be changed (corrupted) to turn one codeword into another.
� The hamming distance from:
– [0101] to [0110] is 2 bits
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 29
– [0101] to [0110] is 2 bits
– [1011101] to [1001001] is 2 bits
– “butter” to “ladder” is 4 characters
– “roses” to “toned” is 3 characters
Another Dot
The code space is now 4.
The hamming distance is still 1.
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 30
Allows:
Error DETECTION for Hamming Distance = 1.
Error CORRECTION for Hamming Distance =1
For Hamming distances greater than 1 an error gives a false correction.
Even More Dots
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 31
Allows:Error DETECTION for Hamming Distance = 2.
Error CORRECTION for Hamming Distance =1.
• For Hamming distances greater than 2 an error gives a false correction.
• For Hamming distance of 2 there is an error detected, but it can not be corrected.
Linear block codes – cont’d
� Error detection capability is given by
� Error correcting-capability t of a code, which is defined as the maximum number of guaranteed correctable errors per
1min −= de
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 32
the maximum number of guaranteed correctable errors per codeword, is
−=2
1mindt
Linear block codes – cont’d
� For memory less channels, the probability that the decoder commits an erroneous decoding is
jnjn
tjM pp
j
nP −
+=
−
≤ ∑ )1(
1
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 33
– is the transition probability or bit error probability over channel.
� The decoded bit error probability is
jnjn
tjB pp
j
nj
nP −
+=
−
≈ ∑ )1(
1
1
p
Linear block codes – cont’d
� Discrete, memoryless, symmetric channel model
Tx. bits Rx. bits
1-p
1-p
p
p
1
0 0
1
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 34
– Note that for coded systems, the coded bits are modulated and transmitted over channel. For example, for M-PSK modulation on AWGN channels (M>2):
where is energy per coded bit, given by
( ) ( )
=
≈MN
REMQ
MMN
EMQ
Mp cbc ππ
sinlog2
log
2sin
log2
log
2
0
2
20
2
2
cE bcc ERE =
� Hamming codes
– Hamming codes are a subclass of linear block codes and belong to the category of perfect codes.
– Hamming codes are expressed as a function of a single integer .
Hamming codes
2≥m
n m 12 :length Code −=
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 35
– The columns of the parity-check matrix, H, consist of all non-zero binary m-tuples.
t
mn-k
mk
nm
m
1 :capability correctionError
:bitsparity ofNumber
12 :bitsn informatio ofNumber
12 :length Code
==
−−=
−=
Hamming codes
� Example: Systematic Hamming code (7,4)
][
1011100
1101010
1110001
33TPIH ×=
=
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 36
][
1000111
0100011
0010101
0001110
44×=
= IPG
Message Word
Code Word Weight of Code word
Message Word
Code Word Weight of Code word
0000 0000000 0 1000 1101000 3
0001 1010001 3 1001 0111001 4
0010 1110010 4 1010 0011010 3
0011 0100011 3 1011 1001011 4
0100 0110100 3 1100 1011100 4
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 37
� Corresponding parity check Matrix
0100 0110100 3 1100 1011100 4
0101 1100101 4 1101 0001101 3
0110 1000110 3 1110 0101110 4
0111 0010111 4 1111 1111111 7
=1110|
0111|
1101|
100
010
001
H
4342143421TPkn
I −
� Minimum distance dmin=3
� Therefore, t=1
� Assuming single-error patterns,
formulate coset leaders:
� Suppose code vector sent: [1 1 1 0 0 1 0]
� Code vector received: [1 1 0 0 0 1 0]
Syndrome Error Pattern
000 0000000
100 1000000
010 0100000
001 0010000
110 0001000
12-Apr-12 The University of Lahore, Islamabad Communication Systems II 38
� Code vector received: [1 1 0 0 0 1 0]
Syndrome is:
110 0001000
011 0000100
111 0000010
101 0000001
[ ]
=
101
111
110
011
100
010
001
0100011s [ ]100=
Adding pattern to received vector yields correct code vector