exercise in the previous class (Apr. 26)

For s = 110010111110001000110100101011101100 (|s|=36),compute 2-values of s for block length with 1, 2, 3 and 4.

n = 1: 0 and 1 are expected to appear 18 times each.s contains |0| =17, |1| = 19, 2 = 12/18+ 12/18=1/9

n = 2: four patterns × 4.5 times.s contains |00| =5, |01| = 1, |10| =6, |11| = 6,2 = 0.52/4.5 + 3.52/4.5 + 1.52/4.5 + 1.52/4.5 = 3.78

n = 3: 2 = 2.67, n=4: 2 = 14.11

Implement the linear congruent method as computer program. see the excel file on http://apal.naist.jp/~kaji/lecture/

1

chapter 3:coding for noisy communication

2

about this chapter

coding techniques for finding and correcting errors

coding in the Chapter 2 ... source coding ( 情報源符号化 )work next to the information sourcegive a compact representation

coding in this chapter ... channel coding ( 通信路符号化 )work next to the communication channelgive a representation which is robust against errors

3

4

two codings

channel

source and receiver sender receiver

source coding encode decode

protection (optional) encrypt decrypt

channel coding encode decode

today’s class

motivation and overviewrephrase of the first day introduction

the models of communication channelsbinary symmetric channel (BSC)

elementary components for linear codes(even) parity check codehorizontal and vertical parity check code

5

communication is erroneous

real communicationno guarantee of “sent information = received information”

radio (wireless) communicationnoise alters the signal waveform

optical disk medium (CD, DVD...)dirt and scratch obstructs correct reading

the difference between sent and received information = error

Errors may be reduced, but cannot be eliminated.

6

error correction in daily life

Three types of errors in our daily life:correctable: “take a train for Ikomo” “... Ikoma”detectable: “He is from Naga city” “Nara” or “Naha”undetectable: “the zip code is 6300102” ???

What makes the difference?

7

IkomaNara

Naha

IkomoNaga

names of places... sparse ( 疎 ) in the set

zipcodes... densely ( 密 ) packed

630010163001026300103

:

trick of error correction and detection

For the error correction, we need to create sparseness artificially.( 人工的に )

phonetic code: Alpha, Bravo, Charlie...あさひの「あ」，いろはの「い」 ...

8

A B CD E FG H I

characters... densely packed

phonetic codes... sparse

Alpha

Bravo

To create the sparseness, we...add redundant sequences,enlarge the space, andkeep representations apart.

in a binary world

assume that you want to send two binary bitswithout redundancy...

9

with redundancy...

Even if you receive “01”, you cannot say if...“01” is the original data, orresult of the modification by errors.

00 01

10 11

00 → 0000001 → 0101110 → 1010111 → 11110

00000

01011

01010

0001100010 encode: to add the redundancy

codewords: results of encodingcode: the set of codewords

the principle of error correction

the sender and the receiver agree the codesender: choose one codeword c and send itreceiver: try to estimate the sent codeword c’

from a received data r which can contain errors

assumption: Errors do not occur so frequently.

with this assumption,error correction ≈ search of the codeword c’ nearest to r

10

00000

0101100010

is it really good?

assume that a symbol is delivered correctly with p=0.9

11

00 → 0000001 → 0101110 → 1010111 → 11110

without codingcorrectincorrect

0.92 = 0.811 – 0.81 = 0.19

with codingcorrect = 0 or 1 error in the five bits

0.95 ＋ 5C10.94 ･ 0.1 = 0.9072

incorrect 1 – 0.9072 = 0.0928

Good for this case, but not always... construction of good codes is the main subject of this chapter

the model of channels

communication channelprobabilistic model with one input and one outputthe inputs and outputs are symbols (discrete channel)an output symbol is generated for each input symbol

no lost, no delaythe output symbol is chosen probabilistically

12

channelinput(sender)

output(receiver)

example of channels: 1

binary symmetric channel (BSC, 二元対称通信路 )inputs = outputs = {0, 1}input = 0 output = 0 with prob. 1 – p, 1 with prob. p.input = 1 output = 0 with prob. p, 1 with prob. 1 – p.p is said to be a bit error probability of the channel

13

0

1

0

1

p

1 – p

p

1 – p

input

output

BSC is ...memoryless ( 記憶がない )

errors occur independentlystable ( 定常 )

the probability p does not change

example of channels: 2

binary erasure channel (BEC, 二元消失通信路 )inputs = {0, 1}, outputs = {0, 1, X}input = 0 output = 0 with prob. 1 – p, X with prob. p.input = 1 output = 1 with prob. 1 – p, X with prob. p.X is a “place holder” of the erased symbol

14

0

1

0

1

p

1 – p

p

1 – p

input

output

X

0

1

0

1

p

1 – p – qp

1 – p – q

Xqq

(another variation)

example of channels: 3

channels with memorythere is correlation between occurrences of errorsa channel with “burst errors”

15

011010101010010101011010110101010

011010101011011010101010110101010

unstable channelsthe probabilistic behavior changes according to timelong-range radio communication, etc.

channel coding: preliminary

We will consider channel coding such that...a binary sequence (vector) of length k is encoded intoa binary sequence (vector, codeword) of length n.

k < n, code C Vn

a sequence is sometimeswritten as a tupleb1b2...bm = (b1, b2, ..., bm)

computation is binary,component-wise:001 + 101 = 100

16

Vk

Vn

good code?

A code C is a subset of Vn

Vn contains 2n vectorsC contains 2k vectors

Given k and n, there are codes.

Which choice of C is good?powerful enough for error correctioneasy encoding and easy decoding

17

Vk Vn

00011011

000011101110 C

the class of linear codes ( 線形符号 )

linear codes

easy encoding(relatively) easy decoding (error detection/correction)

mathematics help constructing good codesthe performance evaluation is not too difficult

Most codes used today are linear codes.

We study some examples of linear codes (today),and learn the general definition of linear codes (next).

18

(even) parity code

the encoding of an (even) parity code ( 偶パリティ符号 ):given a vector (a1, a2, ..., ak) ∈ Vk

compute p = a1 + a2 + ... + ak, and

let (a1, a2, ..., ak , p) be the codeword of (a1, a2, ..., ak)

19

when k = 3... 000001010011100101110111

00000011010101101001101011001111

p = 0 + 1 + 1 = 0

code C

basic property of the parity code

code length: n = k + 1a codeword consists of...

the original data itself (information symbols)added redundant symbol (parity symbol) systematic code ( 組織符号 )

20

00000011010101101001101011001111

code C

1010information symbols

(bits)parity symbol

(bit)

a vector v of length n is codeword v contains even number of 1s.

parity codes and errors

Even parity codes cannot correct errors.Even parity codes can detect odd number of errors.

21

#errors = #differences between the sent and received vectors

0000sent vector(codeword)

0101received vector

we have two (2) errors

0000 0011

0101

00011001

#errors = even the received vector contains even1s

#errors = odd the received vector contains odd 1s

horizontal and vertical parity check code

horizontal and vertical parity check code (2D parity code)place information symbols in a rectangular form ( 長方形 )add parity symbols in the horizontal and vertical directionsreorder all symbols into a vector (codeword)

22

k = 9, encode (a1, a2, ..., a9)a1 a2 a3

a4 a5 a6

a7 a8 a9

p1

p2

p3

q1 q2 q3 r codeword: (a1, a2, ..., a9, p1, p2, p3, q1, q2, q3, r)

p1 = a1 + a2 + a3

p2 = a4 + a5 + a6

p3 = a7 + a8 + a9

q1 = a1 + a4 + a7

q2 = a2 + a5 + a8

q3 = a3 + a6 + a9

r = a1 + a2 + ... + a9

example of encoding

encode 011100101, and the codeword is 0111001010100101

23

2D codes are systematic codesif k = ab, then the code length is n = ab + a + b + 1

0 1 1 01 0 0 11 0 1 00 1 0 1

0111001010100101information

symbolsparity

symbols

2D codes and errors

A 2D code can correct one-bit error in a codeword.place the symbols in a received vectorcount the numbers of “1” in each row/column

if there is no error... all rows and columns contain even 1sif there is an error...

there is one row and one column with odd number of 1sthe intersecting point is affected by an error

24

010110101 0

0

1 1

1

1

0

0 1even

odd

odd eveneven

even

011110101

(received)

correct the third bit

two-bit errors

What happens if two-bits are affected by errors?

25

real errors

two strange rowstwo strange columns

we cannot decide which has happened...

We know something wrong, but cannot spot the errors.

two-bit errors, another case

26

real errors

no strange rowtwo strange columns

We know something wrong, but cannot spot the errors.

additional remark

Do we need the parity of parity?

27

a1 a2 a3

a4 a5 a6

a7 a8 a9

p1

p2

p3

q1 q2 q3 rcodeword: (a1, a2, ..., a9, p1, p2, p3, q1, q2, q3, r)

Even if we don’t have r,we can correct any one-bit error, but...some two-bit errors cause a problem.

additional remark (cnt’d)

We expect that 2D codes detect all two-bit errors.If we don’t use the parity of parity, then...

28

0 0 00 0 00 0 0

0 0 00 1 10 1 0

codeword codeword0 0 00 1 00 0 0

0 0 00 1 00 1 0

1-bit err.

to thenearestcodeword

1-bit err.

some two-bit errors are not detected,instead, they are decoded to a wrong codeword.

0 0 00 1 00 0 0

to the nearestcodeword

summary of today’s class

motivation and overviewrephrase of the first day introduction

the models of communication channelsbinary symmetric channel (BSC)

elementary components for linear codes(even) parity check codehorizontal and vertical parity check code

29

excersise

In the example of page 11, determine the range of the probability p under which the encoding results in better performance.

30