8b error correction

8/13/2019 8b Error Correction

1/129

DIGITAL COMMUNICATION

Error - Correction

November 15, 2011

A.J. Han Vinck


2/129

Error correction is needed in high tech devices

A.J. Han Vinck, rondheim, 2011 2


3/129

fundamental limits set by Shannon (1948)

Bell Labs, 1955.

A.J. Han Vinck, rondheim, 2011 !

Fundamental problem: reproducing at the receiver a message selected at the

transmitter

Shannons contribution: - bound on the efficiency (capacity)

- ho to achieve the bound

The Bell System Technical Journal, !ol. "#, pp. $#9%&"$, '"$%'5',

ctober, 19&.


4/129

A.J. Han Vinck 4

"vervie# o$ the %ect&re

' he (osition o$ channe% coding in a comm&nication chain

' )ome channe% mode%s

' *&rst error mode%s

' +etection r&%es' ntro to the coding (rob%em

' (er$ormance

' he c%ass o$ %inear codes

' *o&nds on the minim&m distance' Eam(%es o$ codes

' h/ error correction


5/129

A.J. Han Vinck 5






' (er$ormance





6/129

A.J. Han Vinck 6

he comm&nication mode%

so&rce

datared&ction

com(ression

data(rotection

sink

essageconstr&ction

decoder

k

n

k

34

34


7/129

A.J. Han Vinck 7

oint-to-(oint

transmitter receiverchanne%

(h/sica%modem modem

message

bits

)igna%

generator

)igna%

(rocessor

bits

message


8/129

A.J. Han Vinck 8

osition o$ Error Contro% Coding

signal

genera!rc"annel #eec!r

k in$%

&is

k !%$%

&is

c"annelk

c"annelk in$%

&is

k !%$%

&is

k in$%

&is

signal

genera!r

c!#e# signal genera!r

#eec!r

#eec!r'#ec!#er

n in$%

&isn

(CC (CC

c!#ing

c!#e# )!#%lai!n


9/129

A.J. Han Vinck *

transmission model (OSI)

Data Link

Control

Data Link

Control

Physial Physial

link

6nre%iab%e trans-mission o$ bits

ransmission o$re%iab%e (ackets


10/129

A.J. Han Vinck +,

)omething to think abo&t

message

bits

bits

message

message

com(ression (rotection o$ bits

correction o$ incorrect bits decom(ression message

Error correctionE7, JE7, etc

Com(ression red&ces bit rate

rotection increases bit rate


11/129

A.J. Han Vinck ++


' he (osition o$ channe% coding in a comm&nication chain' Some hannel models' *&rst error mode%s

' +etection r&%es' ntro to the coding (rob%em' he c%ass o$ %inear codes' *o&nds on the minim&m distance' Eam(%es o$ codes



12/129

A.J. Han Vinck +-

emor/%ess channe%

In$% /0123 !%$%

ransii!n $r!&a&iliies

memor/%ess8

- o&t(&t at time i de(ends on%/ on in(&t at time i

- in(&t and o&t(&t a%(habet $inite


13/129

A.J. Han Vinck +

Eam(%e8 binar/ s/mmetric channe%

9*)C:

(rr!r !%rce

E

X

O%$%In$%

EXY =

E is the binar/ error se;&ences.t. 91: < 1-90: < (

= is the binar/ in$ormationse;&ence

> is the binar/ o&t(&tse;&ence

+9$

, , $

: +

+9$


14/129

A.J. Han Vinck +4

$rom A7N to *)C

{ }EX + '

-

-

-

--+0

n

N enp=

Home#ork8 ca%c&%ate the ca(acit/ as a $&nction o$ A and 2

(

7a&ssian noise #ith (rob. densit/ $&nction

= < ?- A > < = ? N

+ecide8 ? or -

---

-

--

+

A

A

edxepx

=


15/129

A.J. Han Vinck +5

"ther mode%s

,

+

, 0lig" !n

+ 0lig" !;;

$

+9$

/0


16/129

A.J. Han Vinck +6

the eras&re channe%

A((%ication8 cdma detection, disk arra/s

,

+

,

+

+9e

e

e

3 1

/0 3 < , < + = /0 3 < + < /,

+isk 1

+isk 2

+isk !

+isk

+isk 5

3no#n(osition o$error


17/129

A.J. Han Vinck +7

Brom 7a&ssian to binar/ to eras&re

e

xi= +/-

O%$%In$%

yi= xi+ e?

- -

?

?-

?

- -

?

?-EEE


18/129

A.J. Han Vinck +8

Eras&re #ith errors

,

+

,

+(

( e

e

1-(-e

1-(-e


19/129

A.J. Han Vinck +*




' !urst error models


' (er$ormance





20/129

A.J. Han Vinck -,

ode%ing8 net#orking

AckNack

- 1-error ca&ses retransmission

- %ong (ackets a%#a/s have an error

- short (ackets #ith ECC give %o#er e$$icienc/

(acket

)&((ose that a (acket arrives correct%/ #ith (robabi%it/ .

hat is then the thro&gh(&t as a $&ntion o$


21/129

A.J. Han Vinck -+

A )im(%e code

' Bor %o# (acket %oss rates 9e.g. 5D:, sending d&(%icates ise(ensive 9#astes band#idth:

' =" code

F =" a gro&( o$ data (kts together to (rod&ce re(air (ktF ransmit data ? ="8 can recover 1 %ost (kt

10101 11100 00111 11000 10110

10101 11100 0011111000 10110


22/129

A.J. Han Vinck --

CD DVD Blue Laser

Density inreases sensiti"ity


23/129

A.J. Han Vinck -

)!#eling

Ho# to mode%scratches on a

C+

Ans#er isim(ortant $orthe design o$ECC


24/129

A.J. Han Vinck -4

M!#eling> &inar1 rans)issi!n

channe%test

se;&ence

error

se;&ence

rob%em8 0 1 0 0 0 1 0 1 0 0 0 0 1 1 1 '''

+etermination o$ b&rst and g&ard s(ace

b&rst g&ard b&rst


25/129

A.J. Han Vinck -5

A sim(%e error detection method

r!? $ari1

@ill r!? ?ise

Trans)i c!l%)n ?ise

RESULT:RESULT:an1 &%rs !; leng"

L can &e #eece#

L

, + + , , + + , , + +, , , + + , , + + , ,

, , , , , , , , , , ,

, + + , , + + , , , ,, , , + + , , + + + +

, , , , , , , , , + +

hatha((ens #ith b&rsts o$ %ength %arger than G


26/129

A.J. Han Vinck -6

b&rst error mode% 97i%bert-E%%iot:

(rr!r !%rce

#andom#andomerror channe% o&t(&ts inde(endent

/0, < +9 /0+

!urst!ursterror channe% o&t(&ts de(endent

(rr!r !%rce/0, 2 sae < &a# < /0+2sae < &a# < +'-

/0, 2 sae < g!!# < + 9 /0+2sae < g!!# < ,.***

)tate in$o8good or bad

g!!# &a#

transition (robabi%it//g&

/&g

/gg/&&


27/129

A.J. Han Vinck -7

;&estion

g!!# &a#

/g&

/&g

/gg/&&

/0, 2 sae < &a# < /0+2sae < &a# < +'-

/0, 2 sae < g!!# < + 9 /0+2sae < g!!# < ,.**

B"a is "e aerage ;!r /0, ;!r> /gg < ,.* /g& < ,.+ /&g < ,.** /&& < ,.,+ E

In#icae"!? 1!% can 1!% e3en# "e )!#elE


28/129

A.J. Han Vinck -8

Britchman mode%$or mobi%e radio channe%s

multi$le states % and only one state !

Closer to an atual real&'orld hannel

Gn F

+9$

(rr!r $r!&a&ili1 , (rr!r $r!&a&ili1 h

G+


29/129

A.J. Han Vinck -*

Eam(%e 9$rom imo 3orhonen, He%sinki:

' n $ading channe%s received data can e(erience b&rst errors that destro/%arge n&mber o$ consec&tive bits. his is harm$&% $or channe% coding

' nter%eaving distrib&tes burst errorsa%ong data stream

' A (rob%em o$ inter%eaving is

introd&ced etra de%a/' Eam(%e be%o# sho#s b%ock

inter%eaving8time

received

power

Reception after

fading channel

1 0 0 0 1 1 10 1 0 1 1 1 0

0 0 1 1 0 0 1

1 0 0 0 1 1 10 1 0 1 1 1 0 0 0 1 1 0 0 1

1 0 0 0 1 0 0 0 1 0 1 1 11 0 11 0 10 1

Received interleaved data:

Block deinterleaving :

Recovered data:


30/129

A.J. Han Vinck ,

eam(%e

F Consider the code C < I 000, 111

F A b&rst error o$ %ength ! can not be corrected.

F Get &s &se a b%ock inter%eaver !=!

A+ A- A F+ F- F C+ C- C

2 errors

A+ A- A F+ F- F C+ C- C

Interleaver

A+ F+ C+ A- F- C- A F C

A+ F+ C+ A- F- C- A F C

Deinterleaver

A+ A- A F+ F- F C+ C- C

1 error 1 error 1 error


31/129

A.J. Han Vinck +

idd%eton t/(e o$ b&rst channe% mode%

)e%ect channe% k#ith (robabi%it/9k:

ransition(robabi%it/ 90:

0

1

0

1

channe% 1

channe% 2

channe% k hastransition(robabi%it/ (9k:


32/129

A.J. Han Vinck -

m(&%sive Noise C%assi$ication

0a ingle ransien )!#el

arameters o$ sing%e transient 8

-(eak am(%it&de- (se&do $re;&enc/ $0


33/129

A.J. Han Vinck

nter%eaving8 $rom b&rst/ to random

essage inter%eaver channe% inter%eaver -1 message

encoder decoder

b&rst/

Lrandom errorM

oteinter%eaving brings encoding and decoding de%a/

Home#ork8com(are the b%ock and convo%&tiona% inter%eaving #.r.t. de%a/


34/129

A.J. Han Vinck 4

nter%eaving8 b%ock

Channe% mode%s are di$$ic&%t to derive8

- b&rst de$inition

- random and b&rst errors $or (ractica% reasons8 convert b&rst into random error

read in ro# #ise

transmitco%&mn #ise

1

0

0

1

1

0

1

0

0

1

1

0

0

0

0

0

0

1

1

0

1

0

0

1

1


35/129

A.J. Han Vinck 5

+e-nter%eaving8 b%ock

read in co%&mn

#isethis ro# contains 1 error

1

0

0

1

1

0

1

0

0

1

1

e

e

e

e

e

e

1

1

0

1

0

0

1

1

read o&tro# #ise


36/129

A.J. Han Vinck 6

nter%eaving8 convo%&tiona%in(&t se;&ence 0

in(&t se;&ence 1 de%a/ o$ b e%ements

in(&t se;&ence m-1 de%a/ o$ 9m-1:b e%ements

Eam(%e8 b < 5, m < !

in

o&t


37/129

A.J. Han Vinck 7

(3a)$leUMT T%r&! (nc!#er

' Brom E) ) 125 212 v!..0 92000-0:, 6) &%ti(%eing and channe% coding' +ata is segmented into b%ocks o$ G bits, #here 0 G 511

U$$er

C

(nc!#er

L!?er

C

(nc!#er

Inerleaer

1se)aic

O%$%k

Uninerleae#

/ari1

Kk

Inerleae#

/ari1

Kk

In$%k

Inerleae#

In$%

k

O%$%


38/129

A.J. Han Vinck 8

6) nter%eaver8nserting +ata into atri

' +ata is $ed ro#-#ise into a b/ C matri. < 5, 10, or 20, O C 25KF $ G P C then matri is (added #ith d&mm/ characters.

+ - 4 5 6 7 8

* +, ++ +- + +4 +5 +6

+7 +8 +* -, -+ -- - -4

-5 -6 -7 -8 -* , + -

4 5 6 7 8 * 4,

- 6 5 7 4 + 8

+, +- ++ +5 + +4 * +6

+8 -- -+ - +* -, +7 -4

-6 -8 -7 + -* , -5 -

4, 6 5 * 7 8 4Daa is $er)%e# withineac" r!?.

4, 6 5 * 7 8 4

-6 -8 -7 + -* , -5 -

+8 -- -+ - +* -, +7 -4

+, +- ++ +5 + +4 * +6

- 6 5 7 4 + 8

!?s are $er)%e#

4, 6 5 * 7 8 4

-6 -8 -7 + -* , -5 -

+8 -- -+ - +* -, +7 -4

+, +- ++ +5 + +4 * +6

- 6 5 7 4 + 8

Daa is rea# ;r!) )ari3 c!l%)n9?ise


39/129

A.J. Han Vinck *





' Detetion rules' ntro to the coding (rob%em

' (er$ormance





40/129

A channe% transition mode%

A.J. Han Vinck 4,

n(&t = channe% transition "&t(&t >

1 /1

* /*

+++

9/1Q1:

9/*Q1:9/Q1:

9/Q,:

9/1Q,:


41/129

A.J. Han Vinck 4+

-rror $robability (,.P)

)&((ose $or a received vector / #e assign the message i as being transmitted.

hen, the (robabi%it/ o$ a correct decision


42/129

A.J. Han Vinck 4-

aim&m Gike%ihood 9G: receiver

$ind i that maimiSes

9 iQ /: < 9 i, / : 9 / :

< 9 / Qi: 9 i: 9 / :

$or e;&a%%/ %ike%/ ithis is e;&iva%ent to $ind

maim&m 9 / Q i:


43/129

A.J. Han Vinck 4

eam(%e

Bor ( < 0.1 and

=1< 9 0 0 : 9 =1 < 1! :=2< 9 1 1 : 9 =2 < 2!:

7ive /o&r A and G decision $or > < 9 0 1 :


44/129

A.J. Han Vinck 44





' +etection r&%es' Intro to the odin0 $roblem

' (er$ormance





45/129

A.J. Han Vinck 45

*it (rotection

' "btained b/ Error Contro% Codes 9ECC:

F Bor#ard Error Correction 9BEC:

F Error +etection and $eedback 9A:

' er$ormance de(ends on error statisticsT

F Error mode%s are ver/ im(ortant


46/129

A.J. Han Vinck 46

eam(%e

ransmit8 0 0 0 or 1 1 1

Ho# man/ errors can #e correct Ho# man/ errors can #e detect

ransmit8

A < 00000 * < 01011 C < 10101 + < 11110

Ho# man/ errors can #e correct Ho# man/ errors can #e detect

hat is the di$$erence


47/129

ractica% comm&nication s/stem design

messageestimate

channe% decoder

n

Code#ord in

receive

here are 2k code #ords o$ %ength nk is the n&mber o$ in$ormation bits transmitted in n channe% &ses

2k

Code book

Code book#ith errors


48/129

Channe% ca(acit/

+e$inition8

he rate o$ a code is the ratio kn, #here

k is the n&mber o$ in$ormation bits transmitted in n channe% &ses

)hannon sho#ed that8 8

$or Ca(acit/

an encoding methods eist#ith decoding error (robabi%it/ 0


49/129

Encoding and decoding according to )hannon

Code82kbinar/ code#ords #here (90: < 91: < U

Channe% errors8 90 1: < 91 0: < (

i.e. error se;&ences 2nh9(:

+ecoder8search aro&nd received se;&ence $or code#ord

#ith n( di$$erences

s(ace o$ 2n binar/ se;&ences


50/129

decoding error (robabi%it/

1. Bor t errors8 Qtn-(QW X

0 $or n 9%a# o$ %arge n&mbers:

2. W 1 code #ord in region 9code#ords random:

nan#

4$0"+n

kJ;!r

,--

-

-4+-04+0/

4JF7CC0n4J4$0"+0n

n

4$0n"k


51/129

A.J. Han Vinck 5+

A $ic!rial ie?

-nec!rs -kc!#e ?!r#s


52/129

A.J. Han Vinck 5-

decoder

' Com(are received #ord #ith a%% (ossib%e code#ords

code #ords

received

:Dec!#e "e c!#e ?!r# ?i" )ini)%) !; #i;;erences 0M!s Likel1


53/129

A.J. Han Vinck 5

e3a)$le

code #ords8 0 0 0 0 0 0 1 0 1 1 1 0 1 0 1 1 1 1 1 0

received8 0 0 0 1 1

di$$erence8 0 0 0 1 1 0 1 0 0 0 1 0 1 1 0 1 1 1 0 1

best g&ess8 0 1 0 1 1!nl1 + #i;;erence


54/129

A.J. Han Vinck 54

?e "ae s!)e $r!&le)s

' a((ing $rom in$ormation to code #ordsF generation o$ code #ords 9m&t&a%%/ $ar a(art:F storing o$ code book 92k code #ords, %ength n:

' +ecodingF Com(are a received #ord #ith a%% (ossib%e code #ords


55/129

A.J. Han Vinck, rondheim, 2011 55

hat are the c%assica% research (rob%ems

*odeboo+ maimie - # of codewords of

length n

- minimum distance

dmin

*hannel characteriation-/ypes of errors0 memory in the noise behavior -*hannel *apacity

2ecoder-2esign of a decoding3decision algorithm-4inimie complexity


56/129

A.J. Han Vinck 56






' $erformane





57/129

A.J. Han Vinck 57

Definitions

: Ha))ing #isance &e?een 3 an# 1 is

#H>< #03 1 is "e !; $!sii!ns ?"ere 3

i1

i

: T"e )ini)%) #isance !; a c!#e C is

= #)in

< )in #03 1 2 3 C 1 C 3 1P

: Ha))ing ?eig" !; a ec!r 3 is

9 ?03>< #03 , is "e !; $!sii!ns ?"ere 3

i,


58/129

A.J. Han Vinck 58

e3a)$le

: Hamming distance #0 +,,+ ,+++ <

: Minimum distance 0+,+ ,++ ++, < -

: Hamming weight ?0,++,+,+ < 4

Ha))ing ?as a ;a)!%s scienis ;r!) Fell9la& an#

inener !; "e Ha))ing c!#e.


59/129

A.J. Han Vinck 5*

Performance

A c!#e ?i" )ini)%) #isance #)in

is ca$a&le !;

c!rrecing err!rs i;

#)in

- +.

/r!!;> I; err!rs !cc%r "en since #)in

- +

an inc!rrec c!#e ?!r# "as a leas + #i;;erences?i" "e receie# ?!r#.


60/129

A.J. Han Vinck 6,

$ic%re

-+ #i;;erences

A F

#i;;erences ;r!) A #i;;erences ;r!) F


61/129

A.J. Han Vinck 6+

Performane

A code #ith minim&m distance dmin is ca(ab%e o$ correcting E eras&res i$

dmin W E.

roo$8

$ E P dmineras&res occ&r, then at %east 1 (ositionis %e$t to disting&ish bet#een an/ t#o code#ords.

Note8 an eras&re is a (osition #here the receiver kno#s that an error occ&red.


62/129

A.J. Han Vinck 6-

Performane

A code #ith minim&m distance dmin is ca(ab%e o$ correcting E eras&res

and t errors i$

dmin W 2t ?E.

roo$8he minim&m distance is red&ced at maim&m b/ the va%&e E.Hence, i$

dmin - E W 2t

#e can sti%% correct t errors


63/129

A.J. Han Vinck 6

Performane for the %ausian hannel

?-

?

- -

?

E

9eras&re:9error:

(E

(E

'Bor an error correcting code8 2t < 2 n 9(

2

?(!

: error correcting

'ith eras&res8 dmin W n 9(2?(1: ? 2 n (!

')ince (2 P (1 error correction #itho&t eras&res is a%#a/s betterT

(!

(!

(1(2

(!


64/129

A.J. Han Vinck 64






' (er$ormance

' 1he lass of linear odes

' *o&nds on the minim&m distance

' Eam(%es o$ codes



65/129

A.J. Han Vinck 65

GNEA C"+E)

*inar/codes are ca%%ed %inear i$$

Z the com(onent #ise mod&%o-2 s&m o$ t#o code#ords is again a code #ord.

Conse;&ent%/, the all 2ero 'ord is acode #ord.


66/129

A.J. Han Vinck 66

GNEA C"+E 7ENEA"

T"e c!#e ?!r#s are

9 linear c!)&inai!ns !; "e r!?s !; a &inar1 genera!r )ari3 G?i"#i)ensi!ns k n

9 G )%s "ae rank kQ

(3a)$le> C!nsi#er k < n < 6.

+ , , + + ,

genera!r )ari3 G = + + , , + +

+ , + + , +

0+,+G < 0 , , + , + +


67/129

A.J. Han Vinck 67

1se)aic c!#es

Le in general "e )ari3 G &e ?rien as

+ , , 2 + + ,

G < R Ik/ S G< , + , 2 + , +, , + 2 , + +

k < n < 6

T"e c!#e generae# is= linear s1se)aic

= "as )ini)%) #isance .

= "e e;;icienc1 !; "e c!#e is '6.


68/129

A.J. Han Vinck 68

Eam(%e 9o(tim&m:

' )ing%e arit/ check code dmin< 2, k < n-1

100 0 1

0100 0 1

7 < R In9+

/ S


69/129

A.J. Han Vinck 6*

Eam(%e 9o(tim&m:

' e(etition code8 dmin< n, k < 1

' 7 < [ 1 1 1 \


70/129

A.J. Han Vinck 7,

E;&iva%ent codes

' An/ %inear code generator can be bro&ght in Ms/stematic $orm]

' 7s/sDeer)ine "e )ini)%) #isance


74/129

A.J. Han Vinck 74

Linear c!#e generator

9=: re(resents the k bit in$o vector 9 i0, i1, ..., ik-1 :

g9=: is a binar/ (o%/nomia% o$ degree 9 n-k :

3-

the code vector C o$ %ength n can be described b/

C9=: < 9=: g9=: a%% o(erations mod&%o-2.


75/129

A.J. Han Vinck 75

EX: ! "# n ! $ and %&X' ! 1 ( X ( X)

: @!r "e in;!r)ai!n ec!r 0+,+,

C0 < 0+ - 0+ 6

< + - 50+++ , ,+ ,.

: "e enc!#ing $r!ce#%re in 0k 3 n )ari3 ;!r)>

+ + , + , , ,G < , + + , + , , c < I G

, , + + , + ,

, , , + + , +


76/129

A.J. Han Vinck 76

I)$le)enai!n ?i" a s"i;9regiser

T"e ;!ll!?ing s"i; regiser can &e %se#> g0 < 0+

ik9+

...i-i+i,

H!)e?!rk>

gie a #escri$i!n !; "e s"i; c!nr!l ! !&ain "e res%l


77/129

A.J. Han Vinck 77

)ome remarks

' 7enerators $or di$$erent k and n

F are constr&cted &sing mathematics

F %isted in man/ tet books

hat remains is the decodingT


78/129

A.J. Han Vinck 78






' (er$ormance


' !ounds on the minimum distane

' Eam(%es o$ codes



79/129

A.J. Han Vinck 7*

*o&nds on minim&m distance 9Hamming:

' Ginear codes have a s/stematic e;&iva%ent 7

F inim&m Hamming #eight n F k ? 1' 9)ing%eton bo&nd:

' code #ords _ correctab%e error (atterns 2n

,

-

- A ' -

+l! g + 0 4

t p nn

i

nM t n

i

R M h pn

=

=

=


80/129

A.J. Han Vinck 8,

Hamming bo&nd eam(%e

' rob%em87ive the &((er bo&nd o$ the siSe o$ a %inear code C o$ %ength n < K and dmin< !

' )o%&tion8the code corrects &( to one error and th&s QCQ 2K 9 1 ? K: < K ^ QCQ

)ince $or a %inear code QCQ < 2k #e have QCQ O

eam(%e8 1 0 0 0 1 1

7 < 0 1 0 1 0 1

0 0 1 1 1 0


81/129

A.J. Han Vinck 8+

*o&nds on minim&m distance 9%otkin:

- Gist a%% code#ords o$ a binar/ %inar code n

2k code#ords

- Ever/ co%&mn m&st have 2k2 ones 9(roo$:9no a%% Sero co%&mn eists:

Conc%&sion8he tota% n&mber o$ ones in the code#ords is n 2k2


82/129

A.J. Han Vinck 8-

*o&nds on minim&m distance 97i%bert:

' )tart8 )e%ect code#ord $rom 2n(ossib%e #ords

' 1. emove a%% #ords at distance P dmin$rom se%ected code#ord

' 2. )e%ect one o$ the remaining as net code#ord

' !. 7oto 1. &n%ess no (ossibi%ities %e$t.

' E)6G8

' home#ork8 sho# that %ogn 1 F h92(: $or dmin-1 < 2t 2(n ( P `

) i n +

,

- n

d

i

M

ni

=


83/129

A.J. Han Vinck 8

(%ot

1

0.5

< %og2n

( tn

1-h9(:

1-h92(:

0

)ing%eton

%otkin


84/129

A.J. Han Vinck 84

Bor Additive hite 7a&ssian Noise Channe%s;&antiSed as binar/ s/mmetric channe%

' Error (robabi%it/ ( e-EsNoF #here Es is energ/ (er transmitted s/mbo%F No the one-sided noise (o#er s(ectra% densit/

' Bor an &ncoded s/stem ( e-EbNo

' Bor a coded s/stem #ith minim&m distance dF decoding errors occ&r #hen b/ 2t?1 W dF nEs < kEb and th&s

(ce-d2 kn EbNo

' COCL6SIO make the fator C 7d* kn 1


85/129

A.J. Han Vinck 85






' (er$ormance



' -:am$les of odes



86/129

A.J. Han Vinck 86

ichard Hamming 9115-1O:

n 1^, Hamming #as one o$ the ear%iest &sers o$ (rimitive com(&ters at *e%% Gaboratories. He #as $r&strated b/ their%ack o$ $&ndamenta% re%iabi%it/. He there$ore (&SS%ed over the (rob%em o$ ho# a com(&ter might check and correct itso#n res&%ts. ithin severa% months Hamming discovered that etra bits co&%d be added to the interna% binar/ n&mbers o$the com(&ter to red&ndant%/ encode n&merica% ;&antities. his red&ndanc/ enab%ed re%ative%/ sim(%e circ&itr/ to identi$/and correct an/ sing%e bit that #as bad #ithin the encoded b%ock o$ bits 9t/(ica%%/ one #ord o$ data:. his encodingscheme, no# kno#n as Hamming Code, a%so detects the condition o$ an/ t#o bits in the encoded b%ock that $ai%sim&%taneo&s%/.n 150, Hamming (&b%ished his #ork as Error +etecting and Error Correcting Codes in the *e%% )/stem echnica%Jo&rna%, vo%. 2, ((. 1^-1K0.

5 K

^

1 2!

1, 2, ..., ^ are binar/ va%&es 90 or 1:

1, 2, !, are in$ormation bits

5 makes 1?!??5 even

K makes 2?!??K even

^ makes 1?!?2?^ even

1, 2, ..., ^ are transmitted 9stored:

.ny sin0le bit han0e an be orreted


87/129

A.J. Han Vinck 87

)im(%er code, same (rinci(%e

5

K

1

2

!

in$o check

1 ; ;

1 2 ! 5 K

in$o check

1 0 0 0 1 1 transmit1 2 ! 5 K

+ecode received

1 0 0 0 ; 1

8 ransmission e$$icienc/


88/129

A.J. Han Vinck 88

he net code has %ength 15 and 11 in$ormation bits

10

5

2

K

!

O

8 Check that 1 error can be corrected

8 ransmission e$$icienc/

^

11

(

12

!

1


89/129

A.J. Han Vinck 8*

h/ making things com(%icated

' he 9!,1: code

/+ /-+


90/129

A.J. Han Vinck *,

' Can #e do a genera% (resentation >E)

0+-6


91/129

A.J. Han Vinck *+

Hamming codes

: Mini)%) #isance

: C!nsr%ci!n

=G < Ik

All )9%$les !; Ha))ing ?eig" W +

= ?"ere k < -)

= ) 9 +

: *+ec t+at t+e minim,m distance is )-

: .ive t+e efficiency of t+e code


92/129

A.J. Han Vinck *-

Eam(%e k < , n < ^, m < !

+,,, ++,

,+,, +,+G c"!!se $ara)eers "a can &e i)$le)ene#


96/129

A.J. Han Vinck *6

he (arit/ check matri

' e sa# that C < 7 7 is k n7H< 0His n n-k

ro(osition8 "n%/ $or code#ords CH< 0

roo$8 take the s/stematic encoder (resentation

- there are 2n-kdi$$erent s/ndromes

- 2k

di$$erent vectors ca&se the same s/ndromenH< ) mH< ) 9n m:H< 0 i.e. the/ di$$er b/ a code#ord

- the 2k code#ords give the s/ndrome < 0


97/129

A.J. Han Vinck *7

he (arit/ check matri

' A (ro(ert/ to be &sed %ater8

F An/ dmin-1 ro#s o$ Hare %inear%/ inde(endent

eason8 $ not, then %ess than dminro#s give the s/ndrome 0

"n%/ Code#ords give s/ndrome 0 and have minim&m #eight dmin


98/129

A.J. Han Vinck *8

Varshamov-*o&ndLet us onstrut a matri: 3'ith dimensions n : n&k for a 0i"en minimum distane d

)

k in;!r)ai!n s1)&!ls n9k c"eck s1)&!ls

Properties: )ini)%) #isance < n9k+ 0s1)&!ls

leng" -) 9+


101/129

A.J. Han Vinck +,+

7enera% remarks on ) codes

' he genera% (rob%em is the decoding

F ) codes can be decoded &sing' E&c%ids a%gorithm

' *er%ekam( asse/ a%gorithm


102/129

A.J. Han Vinck +,-

eed-&%%er based codes 96):

' )tarting code m < 1 C < I 00,01,10,11 F has minimum distane *m&1

F *m=1ode 'ords 6 of len0th *m

' NE code #ith 2m?2 code #ords o$ %ength 2m?1F 96,6: and 96,6:

distance [ 96,6:, 96,6:\ < 2m #h/ Convince /o&rse%$T

distance [ 96,6:, 9V,V:\ < 2 _ 2m-1 < 2m

distance [ 96,6:, 9V,V:\ < 2 _ 2m-1 < 2m 9&se com(%.(ro(.:


103/129

A.J. Han Vinck +,


)&((ose #e have a generator 7 $or certain m

he constr&ction o$ 7 $or m?1 is sim(%e

Eam(%e8m < 1 m


104/129

A.J. Han Vinck +,4


' *asis $or constr&ction

' ake 2 %inear code generators 71 and 72

' o$ %ength n' and minim&m distance +1 and +2, res(ective%/

' hen, 71 717 < has d

min

< minI2+1,+2

0 72' roo$T


105/129

A.J. Han Vinck +,5

)*N n&mbering code

' $or the )*N n&mbers 9a1, a2, , a10: #e have

#here

' e &se a s(ecia% s/mbo% = in case a10< 10.

' A sing%e transmission error gives as a res&%t

since 11 is a (rime n&mber.

otecheck that the code can a%so detect a do&b%e trans(osition 9interchange o$ 2 n&mbers:.

++)!#,ia+,

+i

i==

.++)!#,aiaa+,ia +,

*

+i

i+,

*

+i

i ==+ ==

D++)!#,XeXeia XX

+,

+i

i =+=


106/129

A.J. Han Vinck +,6

Varshamov-enengo%S codes

' $or the binar/ n&mbers 91, 2, , n: #e have

' A sing%e transmission error gives as a res&%t

' Check that a%% n&mbers are di$$erent and th&s #e can correct 1 errorT

' he cardina%it/ o$ the code

' 3ome'orkcom(are the rate o$ the code #ith the rate o$ the Hamming code.

4+n-)!#0ai3n

+i

i +==

+n-)!#0X'aXei3 X

n

+ii ++=+

=

+n-

-2C2

n

+


107/129

A.J. Han Vinck +,7






' (er$ormance



' Eam(%es o$ codes' >hy error orretion?


108/129

A.J. Han Vinck +,8

h/ error correction

' )/stems #ith errors can be madea%most error $ree

F C+, +v+ #o&%d not #ork #itho&t ) codes


109/129

A )%li %ser a$$licai!n

: (NCODING> ?! %sers rans)i + !r = +

9 %$$!se !ne %ser %ses an err!r c!rrecing c!#e a rae Y.

9 T"e sec!n# %ser X%s sen#s "is in;!r)ai!n a rae +.

: CHANN(L> a##s "e al%es an# "%s "e receier !&seres - , !r 9-

: D(COD(> #ec!#es "e in;!r)ai!n ;!r %ser + %sing an eras%re #ec!#ing

$r!ce#%re A;er #ec!#ing "e in;!r)ai!n ;r!) %ser - can &e calc%lae#

: UM AT( < +.5 &i'rans)issi!n

: NOT(> Ti)e Diisi!n gies a rae +Q

A.J. Han Vinck +,*


110/129

A )%li %ser a$$licai!n

A.J. Han Vinck ++,

?- 1

?- 1

?- 2 0add

6ser 1 < 12

6ser 2


111/129

A.J. Han Vinck +++

h/ error correction


112/129

A.J. Han Vinck ++-

h/ error correction

' )&((ose8 transmit 9n-k bits: receiver

> < = N9oise: >H NH N = < > add =H add =

= = = =

transmit n bits

Bor n-k < nh9(: #e transmitted n ? nh9(: bits instead o$ 2n

' e$erence8 )%e(ian and o%$


113/129

A.J. Han Vinck ++

h/ error correction

' n A s/stems s/stems co%%a(s can be (ost(onedT

tro&gh(&t

100D

kn D

Channe% error (robabi%it/0 1


114/129

A.J. Han Vinck ++4

Combining error detection-correction

' 719=: 729=: generates a code #ith minim&m distance +2' 719=: generates a code #ith minim&m distance +1 P +2' C9=: < 9=: 719=: 729=: < 9=:719=:

decoding8ste( 18 correct a maim&m o$ 9 +1 F 1 :2errors #ith 719=:ste( 28 detect the remaining errors #ith 719=:729=:

(ro(erties8

1. t9 +1 F 1 :2

errors correctab%e2. t +2 - 1 - 9 +1 F 1 :2 errors detectab%e

Home#ork8 constr&ct some eam(%es


115/129

A.J. Han Vinck ++5

Combining error detection-correction

' Eam(%e8

n k dmin generator (o%/nomia% 9octa%:

K! 5^ ! 71 < 10! 9octa%:

51 5 72 71 < 71 912^:

5 ^ 7! 72 71< 72 71 91^:


116/129

A.J. Han Vinck ++6

Eam(%e8 b%&etooth

' hree error correction schemes are de$ined $or *%&etooth8F 1! rate BEC, a sim(%e re(etitionF 2! rate BEC, a shortened Hamming Code 910,15:F A scheme $or data 9a&tomatic retransmission re;&est:

' BEC schemes red&ce the n&mber o$ retransmissions

' A CC code decides #hether a (acketheader contains errors, i.e.

' transmit C9=: < 9=: g9=:' receive 9=: < C9=: ? E9=:' check 9=: mod&%o g9=: < 0


117/129

Code shortening

' )&((ose that #e have a ) code in s/stematic $orm n4

1 0 ... 0

0 1 ... 0 he generator takes the $irst k ? n4-k co%&mns

arit/ (art minim&m distance is n4-k?1

... 0 0 ... 1

k n4-k

k4 n-k

take the %ast k4 ro#s o$ %ength k4 ? n - k

100...0 arit/ (art minim&m distance is n4 F k4 ? 1

00... 1++7

Case a

Case b


118/129

A.J. Han Vinck ++8

Channe% #ith insertions and de%etions

' *ad s/nchroniSation or c%ock recover/ at receiver8

F insertion

''' 1 0 0 0 1 1 1 0 0 1 0 ''' ''' 1 0 0 10 1 1 1 0 0 1 0 '''

F de%etion

''' 1 0 0 01 1 1 0 0 1 0 ''' ''' 1 0 0 1 1 1 0 0 1 0 '''

rob%em8 $inding start and end o$ messages


119/129

A.J. Han Vinck ++*


' Eam(%e8

F the $o%%o#ing code corrects' a sing%e inversion error

' or a sing%e de%etioninsertion error

' C < I 0 0 0 0 0 0, 1 1 1 1 1 1, 0 0 1 0 1 1, 1 1 0 1 0 0

F dmin< !F An insertionde%etion makes the #ord %ongershorter


120/129

A.J. Han Vinck +-,


' +&e to errors in bit (attern

$%ag < 1 1 1 1 1 0, avoid 1 1 1 1 1 in $rame

''' 0 1 1 1 1 1 0 0 1 1 0 1 ''' ''' 0 1 1 1 1 01 0 0 1 1 0 1 '''

''' 0 1 1 0 1 01 0 0 1 1 0 1 ''' insertion

''' 0 1 1 1 0 0 0 0 1 1 0 1 ''' ''' 0 1 1 1 10 0 0 1 1 0 1 '''

''' 0 1 1 1 1 0 0 1 1 0 1 ''' de%etion


121/129

A.J. Han Vinck +-+

Channe%s #ith inter$erence

' Eam(%e 9o(tica% channe%:

Error (robabi%it/ de(ends on s/mbo%s in neighboring s%ots


122/129

A.J. Han Vinck +--

Channe%s #ith memor/ 9e8recording:

: (3a)$le> i<

i

i9+

i + 9+ P

Xi Xi-1

Yi

{ +2, 0, -2


123/129

,5.,-.+4 +-

he (ass#ord (rob%em

serverPass'ordPass'ord< Honar/Hash9Honar/:

memor/ Hash9Honar/:

Ca%c&%ateHash9(ass#ord:

com(are

server

am8 *ahram

Hash9 :

memor/ Hash9 :

Ca%c&%ateHash9 :

com(are

am8*ahram

biometribiometri


124/129

,5.,-.+4 +-4

rob%em in a&thentication8

(ass#ords8 need to be eact

biometrics8 are on%/ ver/ simi%ar


125/129

,5.,-.+4 +-5

he (rob%em A8 sec&re sketch

** )tore *H)tore *H

9s/ndrome:9s/ndrome:

*< **< * NNca%c&%ateca%c&%ate

9*9* *:H*:H < NH< NH

*H*H

+EC"+E N+EC"+E N**NN

*H*H

@rom NHNH *k$ossible "etors ! e:$


126/129

,5.,-.+4 +-6

A&thentication8 sec&re storage, #ith errors

-nrollment-nrollment

)< * E9:

random

Condition8

given ) and Hash9:

it is hard

to estimate *,

store )

Hash9:

*

E9:

authentiationauthentiation )Hash9:

*4 )*4 < * E9:*4 < E9: N


127/129

,5.,-.+4 +-7

A&thentication8 sec&re storage, #ith errors

attakattak

)Hash9:

7&ess or *

*4 < )

E94: < *

E9:

E94:or

) H< [* E9:\ H< *4 H

-nrollment-nrollment

)< * E9:

random

Condition8

given ) and Hash9:

it is hard

to estimate *,

store )

Hash9:

*

E9:

*i t


128/129

12!

*inar/ entro(/

inter(retation8%et a binaryse;&ence contain (n ones, then#e can s(eci$/ each se;&ence #ith

%og22nh9(:

< n h9(: bits

0 4-nh pn

pn

4$0"

$n

nl!g

n

+li) -n

=

Home#ork8rove the a((roimation &sing %nNT N %nN $or N %arge.

6se a%so %oga < / %ogb < / %ogba

he )tir%ing a((roimation Q - N NN N N e

T+e /inary Entropy> "0$ < $l!g $ 0+ $ l!g 0+ $


129/129

0. 1

0. 2

0. 3

0. 4

0. 5

0. 6

0. 7

0. 8

0. 9

1

h

T+e /inary Entropy> "0$ < 9$l!g-$ = 0+9$ l!g-0+9$

N!e>

"0$ < "0+9$

8b error correction

Documents