
2/17

Basic ConceptsExample: Binary Repetition Codes

(3,1) code: 0 ==> 000 1 ==> 111

Received: 011. What was transmitted?

scenario A: 111 with one error in 1st locationscenario B: 000 with two errors in 2nd & 3rd locations.

Decoding:

P(A) = (1-p)2p

P(B) = (1-p)p2

P(A) > P(B) (forp 111

2


3/17

Probability of Error After Decoding(3,1) repetition code can correct single errors.

In general for a tc-error correcting code:

Bit error probability = [for the (3,1) code, Pb= Pu]

Gain: For a BSC withp= 10-2, Pb=3x10-4.

Cost: Expansion in bandwidth or lower rate.

3

pppP

pppP

C

u

23

32

)1(3)1(11

)1(yprobabiliterrorBlockUndetected

3

3

2

3

init

i

C ppPc

i

n

)1(

0

ub PP


4/17


5/17

Geometrical IllustrationHamming Cube

5

000

111

001

011

101

100

110

010


6/17

Error Correction and DetectionConsider a code consisting of two codewords withHamming distance dmin. How many errors can bedetected? Corrected?

# of errors that can be detected = td= dmin -1

# of errors that can be corrected = tc

=

In other words, for t-error correction, we must have

dmin = 2tc + 1

6

2

1min

d


7/17


8/17

Minimum Distance of a Code Def.: The minimum distance of a code Cis the minimum

Hamming distance between any two different codewords.

A code with minimum distance dmin can correct all errorpatterns up to and including t-error patterns, where

dmin = 2tc + 1It may be able to correct some higher error patterns, butnot all.

8

d d c c c c C i j

i j i jmin min ( , ) and in


9/17

Example: (7,4) Code

9

No. Messa e Codeword No. Messa e Codeword

0 0000 0000000 8 0001 1010001

1 1000 1101000 9 1001 01110012 0100 0110100 10 0101 1100101

3 1100 1011100 11 1101 0001101

4 0010 1110010 12 0011 0100011

5 1010 0011010 13 1011 1001011

6 0110 1000110 14 0111 0010111

7 1110 0101110 15 1111 1111111


10/17

Coding: Gain and Cost (Revisited) Given an (n,k) code.

Gain is proportional to the error correction capability,tc.

Cost is proportional to the number of check digits, n-k= r.

Given a sequence ofkinformation digits, it is desiredto add as few check digits ras possible to correct as

many errors (t) as possible.What is the relation between these code parameters?

Note some text books uses m rather than r for the number check bits

10


11/17


12/17

Hamming Bound (contd) In other words, When a codeword is selected, none of

the n-bit sequences that differ from that codeword bytcor less locations can be selected as a codeword.

Consider the all-zero codeword. The number of words

that differ from this codeword byj locations is

The total number of words in any sphere (including

the codeword at the center) is

12

n

j

cc t

j

t

j j

n

j

n

01

1


13/17


14/17

Hamming Bound (contd) The above bound is known as the Hamming Bound. It provides a

necessary, but not a sufficient, condition for the construction of an(n,k) tc-error correcting code.

Example: Is it theoretically possible to design a (10,7) single-error

correcting code?

A code for which the equality is satisfied is called a perfect code. There are only three types of perfect codes (binary repetition codes, the

hamming codes, and the Golay codes). Perfect does not mean best!

14

310 10

1 10 11 2 .0 1

It is not possible.


15/17

Gilbert Bound While Hamming bound sets a lower limit on the number of

redundant bits (n-k) required to correct tc errors in an (n,k) linearblock code.

Another lower limit is the Singleton bound

Gilbert bound places an upper bound on the number of redundantbits required to correct tc errors.

It only says there exist a code but it does not tell you how to find it.

2

2

0

- logct

j

nn kj

15

min 1d r


16/17

The Encoding Problem How to select 2kcodewords of the code Cfrom the 2n

sequences such that some specified (or possibly themaximum possible) minimum distance of the code isguaranteed?

Example: How were the 16 codewords of the (7,4) codeconstructed? Exhaustive search is impossible, exceptfor very short codes (small kand n)

Are we going to store the whole table of 2k(n+k)entries?!

A constructive procedure for encoding is necessary.

16


17/17

The Decoding ProblemStandard Array0000000 1101000 0110100 1011100 1110010 0011010 1000110 0101110 1010001 0111001 1100101 0001101 0100011 1001011 0010111 11111110000001 1101001 0110101 1011101 1110011 0011011 1000111 0101111 1010000 0111000 1100100 0001100 0100010 1001010 0010110 11111100000010 1101010 0110110 1011110 1110000 0011000 1000100 0101100 1010011 0111011 1100111 0001111 0100001 1001001 0010101 11111010000100 1101100 0110000 1011000 1110110 0011110 1000010 0101010 1010101 0111101 1100001 0001001 0100111 1001111 0010011 11110110001000 1100000 0111100 1010100 1111010 0010010 1001110 0100110 1011001 0`10001 1101101 0000101 0101011 1000011 0011111 1110111

0010000 1111000 0100100 1001100 1100010 0001010 1010110 0111110 1000001 0101001 1110101 0011101 0110011 1011011 0000111 11011110100000 1001000 0010100 1111100 1010010 0111010 1100110 0001110 1110001 0011001 1000101 0101101 0000011 1101011 0110111 10111111000000 0101000 1110100 0011100 0110010 1011010 0000110 1101110 0010001 1111001 0100101 1001101 1100011 0001011 1010111 0111111

Exhaustive decoding is impossible!!Well-constructed decoding methods are required.

Two possible types of decoders:1) Complete: always chooses minimum distance2) Bounded-distance: chooses the minimum distance up

to a certain tc. Error detection is utilized otherwise.

17

4 channel coding _ basic concepts

Documents