lecture 25: channel encoding: block codes block codes linear block code (n,k) – a set with...

EE4723 : Digital Communications II

Lecture 25:

Channel Encoding: Block Codes

12-Apr-12 The University of Lahore, Islamabad Communication Systems II 1

Linear block codes

� Linear block code (n,k)

– A set with cardinality is called a linear block code if, and only if, it is a subspace of the vector space .nV

nVC ⊂ k2

VCV ⊂→


• Members of C are called code-words.• The all-zero codeword is a codeword.• Any linear combination of code-words is a codeword.

nk VCV ⊂→

Linear block codes – cont’d

nVkV

C

mapping


C

Bases of C

� The subset chosen for the code should include as many as elements to reduce the redundancy but they should be as apart as possible to maintain good error performances


Figure 6.10: Linear block-code structure


� The information bit stream is chopped into blocks of k bits.

� Each block is encoded to a larger block of n bits.

� The coded bits are modulated and sent over channel.

� The reverse procedure is done at the receiver.


Data blockChannelencoder Codeword

k bits n bits

rate Code

bits Redundant

n

kR

n-k

c =

Generator Matrix� If k is large, a lookup table implementation of the encoder becomes

prohibitive

� Let the set of 2k codewords {U} be described as:

U=m1V1+ m2V2 + …..+ mkVk

� In general, generator matrix can be defined by the following k x n array:

vvvV L


array:

� Generation of codeword U:

U=mG

=

=

knkk

n

n

k vvv

vvv

vvv

V

V

V

G

L

M

L

L

M

21

22221

11211

2

1

Linear block codes –cont’d

nVkV

C

Bases of C

mapping


– A matrix G is constructed by taking as its rows the vectors on the basis, .

Bases of C

},,,{ 21 kVVV K

=

=

knkk

n

n

k vvv

vvv

vvv

L

MOM

L

L

M

21

22221

112111

V

V

G


� Encoding in (n,k) block code

mGU =mmmuuu

V

V

⋅= 2

1

),,,(),,,( KK


– The rows of G, are linearly independent.

kn

k

kn

mmmuuu

mmmuuu

VVV

V

V

⋅++⋅+⋅=

⋅=

2221121

22121

),,,(

),,,(),,,(

KK

MKK


=

=1

0

0

0

1

0

0

0

1

1

1

0

0

1

1

1

0

1

3

2

1

V

V

V

G

1

1

0

0

0

1

1

0

1

1

1

1

1

0

0

0

1

1

1

1

0

0

0

1

1

0

1

0

0

0

1

0

0

0

1

0

1

0

0

1

1

0

0

1

0

0

0

0

1

0

0

0

1

0Message vector Codeword


� Generate Codeword U4 for the fourth message vector 1 1 0 in Table 6.1

U4 =[ 1 1 0] V=V1+V2+0*V3

= 110100+011010+000000

=101110 ( Codeword for the message vector 110)

1

1

1

1

1

0

0

0

0

1

0

1

1

1

1

1

1

0

101110101


� Systematic block code (n,k)

– A systematic (n,k) linear block code is a mapping a k-dimensional message vector to an n-dimensional code word such that part of the sequence has k message digits and remaining (n-k) are parity digits


digits and remaining (n-k) are parity digits

– For a systematic code, the first (or last) k elements in the codeword are information bits.

matrix )(

matrixidentity

][

knk

kk

k

k

k

−×=×=

=

P

I

IPG

Systematic Linear Block codes

� A systematic linear code will have a generator matrix

=

=

−

−

−

100

010

001

)(,21

)(,22221

)(,11211

LL

MM

LL

LL

M

M

M

knkkk

kn

kn

k

ppp

ppp

ppp

IPG


� Combining the previous equations

×…=

−

−

−

100

010

001

].m,m,[mu,....u,u

)(,21

)(,22221

)(,11211

k2121

LL

MM

LL

LL

knkkk

kn

kn

n

ppp

ppp

ppp

Where

ui = m1p1i+ m2p2i+ ….mkpki for i=1,…(n-k)

= mi-n+k for i=(n-k+1)….n

� Thesystematiccodevectorcanbeexpressedas:




44 344 21K

44 344 21K

bitsmessage

k

bitsparity

kn mmmpppU ,,,,,,, 2121 −=

)(,)(,22)(,11

22221212

12121111

knkkknknkn

kk

kk

pmpmpmp

pmpmpmp

pmpmpmp

−−−− +++=+++=+++=

L

L

L

Example

� For (6,3) code example in sec.6.4.3, the codewords can be described as:

[ ]

=M

M

M

100

010

001

101

110

011

,, 321 mmmU

(6.30)



43421M

43421

3

100101

IP

{ { {654321

321322131 ,,,,,uuuuuu

mmmmmmmmm4342143421321

+++=

(6.30)

(6.31)


� For any linear code we can find a matrix , such that its rows are orthogonal to the rows of G:

nkn ×− )(H

0GH =T


� H is called the parity check matrix and its rows are linearly independent.

� For systematic linear block codes:

][ Tkn PIH −=


� A systematic linear code will have a generator matrix

=

=

−

−

−

100

010

001

)(,21

)(,22221

)(,11211

LL

MM

LL

LL

M

M

M

knkkk

kn

kn

k

ppp

ppp

ppp

IPG


� Combining the previous equations

×…=

−

−

−

100

010

001

].m,m,[mu,....u,u

)(,21

)(,22221

)(,11211

k2121

LL

MM

LL

LL

knkkk

kn

kn

n

ppp

ppp

ppp

Where

ui = m1p1i+ m2p2i+ ….mkpki for i=1,…(n-k)

= mi-n+k for i=(n-k+1)….n





44 344 21K

44 344 21K

bitsmessage

k

bitsparity

kn mmmpppU ,,,,,,, 2121 −=

)(,)(,22)(,11

22221212

12121111

knkkknknkn

kk

kk

pmpmpmp

pmpmpmp

pmpmpmp

−−−− +++=+++=+++=

L

L

L

Example

� For (6,3) code example in sec.6.4.3, the codewords can be described as:

[ ]

=M

M

M

100

010

001

101

110

011

,, 321 mmmU

(6.30)



43421M

43421

3

100101

IP

{ { {654321

321322131 ,,,,,uuuuuu

mmmmmmmmm4342143421321

+++=

(6.30)

(6.31)


� For any linear code we can find an matrix , such that its rows are orthogonal to the rows of G:

nkn ×− )(H

0GH =T


� H is called the parity check matrix and its rows are linearly independent.

� For systematic linear block codes:

][ Tkn PIH −=


FormatChannel encoding

Modulation

Channeldecoding

FormatDemodulation

Detection

Data source

Data sink

U

r

m

m̂

channel

eUr +=


� Syndrome testing:

– S is syndrome of r, corresponding to the error pattern e.

or vectorpattern error ),....,,(

or vector codeword received ),....,,(

21

21

n

n

eee

rrr

==

e

reUr +=

TT eHrHS ==


� Standard array– For row , find a vector in of

minimum weight which is not already listed in the array.

– Call this pattern and form the row as the

kni −= 2,...,3,2nV

ie

th:i


corresponding coset

kknknkn

k

k

22222

22222

221

UeUee

UeUee

UUU

⊕⊕

⊕⊕

−−− L

MOLM

L

Lzero

codeword

coset

coset leaders


� Standard array and syndrome table decoding

1. Calculate

2. Find the coset leader, , corresponding to .

3. Calculate and corresponding .

TrHS =

iee =ˆ S

erU ˆˆ += m̂


– Note that

• If , error is corrected.

• If , undetectable decoding error occurs.

)ˆˆ(ˆˆ e(eUee)UerU ++=++=+=

ee =ˆ

ee ≠ˆ


� Example: Standard array for the (6,3) code

000101110001011111101011101100011000110111000010

000110110010011100101000101111011011110101000001

000111110011011101101001101110011010110100000000

codewords


010110100101010001

010100100000

100100010000

111100001000

000110110111011010101101101010011100110011000100

LL

M

MMM

Coset leaders

coset

� No column of H can be all zeros, or else an error in the corresponding codeword position would not affect the syndrome and would be undetectable

� All columns of H must be unique. If two columns of H were identical, errors in these two corresponding codeword positions would be indistinguishable

Example: Codeword U = 1 0 1 1 1 0 , and r = 0 0 1 1 1 0 Find S=rHT

Requirements of the parity-check matrix


Example: Codeword U = 1 0 1 1 1 0 , and r = 0 0 1 1 1 0 Find S=rHT

[ ]

[ ] [ ]00111,11,1

101

110

011

100

010

001

011100

=++=

=

= TrHS

[ ]

[ ]001

000001

=

=

=

T

T

H

eHS


110000100

011000010

101000001

000000000

:computed is of syndrome The

received. is (001110)

ted. transmit(101110)

===

==

r

r

U

TT

Error pattern Syndrome


111010001

100100000

010010000

001001000

(101110)(100000)(001110)ˆˆ

estimated is vector corrected The

(100000)ˆ

is syndrome this toingcorrespondpattern Error

(100)(001110)

=+=+=

=

===

erU

e

HrHS TT

=100

010

001

][ rrrrrrS

TrHS =� The received signal is multiplied with the parity check matrix:

Decoder implementation


=

101

110

011][ 654321 rrrrrrS

and

][

][

][

6533

5422

6411

rrrs

rrrs

rrrs

++=

++=

++=

Implementation of decoder


Figure 6.12: Implementation of the (6,3) decoder


� The Hamming weight of vector U, denoted by w(U), is the number of non-zero elements in U.

� The Hamming distance between two vectors U and V, is the number of elements in which they differ.


� The minimum distance of a block code is

)()( VUVU, ⊕= wd

)(min),(minmin ii

jiji

wdd UUU ==≠

How it works: 3 dots

Only 3 possible words

Distance Increment = 1

One Excluded State (red)

A B C


It is really a checksum.

� Single Error Detection

� No error correction

A B C

A C

⇒ Two valid code words (blue)

This is a graphic representation of the “Hamming Distance”

Hamming Distance

� Definition:

– The number of elements that need to be changed (corrupted) to turn one codeword into another.

� The hamming distance from:

– [0101] to [0110] is 2 bits


– [0101] to [0110] is 2 bits

– [1011101] to [1001001] is 2 bits

– “butter” to “ladder” is 4 characters

– “roses” to “toned” is 3 characters

Another Dot

The code space is now 4.

The hamming distance is still 1.


Allows:

Error DETECTION for Hamming Distance = 1.

Error CORRECTION for Hamming Distance =1

For Hamming distances greater than 1 an error gives a false correction.

Even More Dots


Allows:Error DETECTION for Hamming Distance = 2.

Error CORRECTION for Hamming Distance =1.

• For Hamming distances greater than 2 an error gives a false correction.

• For Hamming distance of 2 there is an error detected, but it can not be corrected.


� Error detection capability is given by

� Error correcting-capability t of a code, which is defined as the maximum number of guaranteed correctable errors per

1min −= de


the maximum number of guaranteed correctable errors per codeword, is

−=2

1mindt


� For memory less channels, the probability that the decoder commits an erroneous decoding is

jnjn

tjM pp

j

nP −

+=

−

≤ ∑ )1(

1


– is the transition probability or bit error probability over channel.

� The decoded bit error probability is

jnjn

tjB pp

j

nj

nP −

+=

−

≈ ∑ )1(

1

1

p


� Discrete, memoryless, symmetric channel model

Tx. bits Rx. bits

1-p

1-p

p

p

1

0 0

1


– Note that for coded systems, the coded bits are modulated and transmitted over channel. For example, for M-PSK modulation on AWGN channels (M>2):

where is energy per coded bit, given by

( ) ( )

=

≈MN

REMQ

MMN

EMQ

Mp cbc ππ

sinlog2

log

2sin

log2

log

2

0

2

20

2

2

cE bcc ERE =

� Hamming codes

– Hamming codes are a subclass of linear block codes and belong to the category of perfect codes.

– Hamming codes are expressed as a function of a single integer .

Hamming codes

2≥m

n m 12 :length Code −=


– The columns of the parity-check matrix, H, consist of all non-zero binary m-tuples.

t

mn-k

mk

nm

m

1 :capability correctionError

:bitsparity ofNumber

12 :bitsn informatio ofNumber

12 :length Code

==

−−=

−=

Hamming codes

� Example: Systematic Hamming code (7,4)

][

1011100

1101010

1110001

33TPIH ×=

=


][

1000111

0100011

0010101

0001110

44×=

= IPG

Message Word

Code Word Weight of Code word

Message Word

Code Word Weight of Code word

0000 0000000 0 1000 1101000 3

0001 1010001 3 1001 0111001 4

0010 1110010 4 1010 0011010 3

0011 0100011 3 1011 1001011 4

0100 0110100 3 1100 1011100 4


� Corresponding parity check Matrix

0100 0110100 3 1100 1011100 4

0101 1100101 4 1101 0001101 3

0110 1000110 3 1110 0101110 4

0111 0010111 4 1111 1111111 7

=1110|

0111|

1101|

100

010

001

H

4342143421TPkn

I −

� Minimum distance dmin=3

� Therefore, t=1

� Assuming single-error patterns,

formulate coset leaders:

� Suppose code vector sent: [1 1 1 0 0 1 0]

� Code vector received: [1 1 0 0 0 1 0]

Syndrome Error Pattern

000 0000000

100 1000000

010 0100000

001 0010000

110 0001000


� Code vector received: [1 1 0 0 0 1 0]

Syndrome is:

110 0001000

011 0000100

111 0000010

101 0000001

[ ]

=

101

111

110

011

100

010

001

0100011s [ ]100=

Adding pattern to received vector yields correct code vector

lecture 25: channel encoding: block codes block codes linear block code (n,k) – a set with...

Documents