chapter 5 signal-space analysisshannon.cm.nctu.edu.tw/comtheory/chap5.pdf5.2 geometric...

Chapter 5 Signal-Space Analysis

Signal space analysis provides a mathematically elegant and highly insightful tool for the study of data transmission.

© Po-Ning [email protected] Chapter 5-2

5.1 Introduction

o Statistical model for a genetic digital communication systemn Message source: A priori probabilities for information

source

n Transmitter: The transmitter takes the message source output mi and (en-)codes it into a distinct signal si(t) suitable for transmission over the channel. So:

MimPp ii ,...,2,1for )( ==

MitsPmPp iii ,...,2,1for ))(()( ===


5.1 Introduction

o si(t) must be a real-valued energy signal (i.e., a signal with finite energy) with duration T.

n Channel: The channel is assumed linear and with a bandwidth wide enough to pass si(t) with no distortion.o A zero-mean additive white Gaussian noise (AWGN)

is also assumed to facilitate the analysis.

.)(0

2 ¥<= òT

ii dttsE

o We can simplify the previous system block diagram to:

o Upon the reception of x(t) with a duration of T, the receiver makes the best estimate of mi. (We haven’t defined what “the best” means.)


5.1 A Mathematical Model


5.1 Criterion for the “Best” Decision

o Best = Minimization of the average probability of symbol error.

n It is optimum in the minimum-probability-of-error sense.n Based on this criterion, we begin to design the receiver

that can give the best decision.

å=

¹×=M

iiiie mmmPpP

1

)|ˆ(


5.2 Geometric Representation of Signals

o Signal space conceptn Vectorization of the (discrete or continuous) signals

removes the redundancy in signals, and provides a compact representation for them.

n Determination of the vectorization basiso Gram-Schmidt orthogonalization procedure


5.2 Gram-Schmidt Orthogonalization Procedure

Given !v1,!v2,…, !vk, how to find an orthonormal basis for them?

(step i) Let !u1 =!v1

|| !v1 ||.

(step ii) !u2' =!v2 − (!v2 ⋅

!u1)!u1. Set !u2 =!u2

'

|| !u2' ||

.

(step iii) For i = 3, 4,..., Let !ui

' =!vi − (!vi ⋅

!ui−1)!ui−1 − (!vi ⋅!ui−2 )!ui−2 −"− (!vi ⋅

!u1)!u1.

Set !ui =!ui

'

|| !ui' ||

.

(step iv) Then !u1,!u2,", !uk( ) forms an orthonormal basis.


(viii) Cauchy Schwartz inequality : |!v1 ⋅

!v2 |≤|| !v1 || ⋅ || !v2 || with equality holding if !v1 = a

!v2.(xi) norm square : || !v1 +

!v2 ||2=|| !v1 ||2 + || !v2 ||2 +2!v1 ⋅!v2.

(x) Pythagorean property : If orthogonal, || !v1 +

!v2 ||2=|| !v1 ||2 + || !v2 ||2 .(xi) Matrix transformation w.r.t. matrix A : !v1 = A

!v2.(xii) eigenvalues w.r.t. matrix A: solution λ of det A-λI[ ] = 0.(xiii) eigenvectors w.r.t. eigenvalue λ : solution !v of A!v = λ!v.


5.2 Signal Space Concept for Continuous Functions

Properties for continuous functions(i) (complex valued) signal : z(t)

(ii) inner product : < z(t), z(t)>= z(t)z*(t)dta

b∫ .

(iii) orthogonal, if inner product = 0.

(iv) norm : z(t) = | z(t) |2 dta

b∫

(v) orthonormal, if inner product = 0, and indivual norm =1.(vi) linearly independent vectors, if none can be represented as a linear combination of others.(vii) triangle inequality : z(t)+ z(t) ≤|| z(t) ||+ || z(t) || .


o Problem : How to minimize the “energy” of ? )(ˆ)()( tstste -=

Q.E.D.


)(ts

)(1 ty

)(2 ty

)(ˆ ts)(),( 22 ttsa y=

)(),( 11 ttsa y=)(te

o Interpretationn aj is the projection of s(t) onto the Yj(t)-axis.n |aj|2 is the energy-projection of s(t) onto the Yj(t)-axis.


)(ts

)(1 ty

)(2 ty

)(ˆ ts)(),( 22 ttsa y=

)(),( 11 ttsa y=)(te

Slide 5-13 also yields:


5.2 Signal Space Concept for Continuous Functions

o For simplicity, we now focus on real-valued functions.o Completeness

n If every finite energy signal s(t) satisfies

n Example. Fourier series

is a complete orthonormal set.

complete for signals defined over [0,T).



MiTttfstsN

jjiji ,...,2,1 ,0 ,)()(

1

=<£=å=

NjMi

dttftssT

jiij

,...,2,1,,...2,1

,)()(0

==

= ò

lorthonorma }{ 1Niif =



o Through the signal space concept, si(t) (where 1 £ i £ M) can be unambiguously represented by an N-dimensional signal vector (si1, si2,…, siN) over an N-dimensional signal space.

o The design of transmitters becomes the selection of Mpoints over the signal space, and the receivers make a guess about which of the M points was transmitted.

o In the N-dimensional signal space,n length square of the vector = energy of the signaln angle between vectors = energy correlation between

signalsåò=

==N

jij

T

ii sdttsts1

2

0

22 )(||)(||||)(||||)(||

)(),()cos(

tstststs

ki

kiik ×=q



n The length square of a vector and the angle between vectors are independent of the basis used (Note that no translation of the origin is allowed).

o From this view, n the transmitter may be viewed as a synthesizer, which synthesizes the transmitted signal by a bank of Nmultipliers.

n the receiver may be viewed as an analyzer, which correlates (product-integrate) the common input into individual informational signal.


5.2 Geometric Representation of Energy Signals

o Illustration the geometric representation of signals for the case of N = 2 and M = 3


5.2 Euclidean Distance

o After vectorization, we can calculate the Euclidean distancebetween two signals, which is the squared root of:

åò=

-=-=-N

jkjijki

T

ki sststsdttsts1

22

0

2 )(||)()(||))()((


Example 5.1 Schwarz Inequality

o Cauchy-Schwarz inequality and angle between signalsn Cauchy-Schwarz inequality said that

n Also, the angle between signals gives that

n Hence, Cauchy-Schwarz inequality can be equivalently stated as:

s1(t), s2 (t)2≤|| s1(t) ||2 ⋅ || s2 (t) ||2 with equality holding if s1(t) = cs2 (t).

||)(||||)(||)(),(

)cos(21

2112 tsts

tsts×

=q

| cos(θ12 ) |2≤1 with equality holding if θ12 = 0 or π


5.2 Basis

o The (complete) orthonormal basis for a signal space is not unique!n So, the synthesizer and the analyzer for the transmission

of the same informational messages are not unique!o One way to determine a set of orthonormal basis is the

Gram-Schmidt orthogonalization procedure.

o Try and practice Example 5.2 yourself!


5.3 Conversion of the Continuous AWGN Channel to a Vector Channel

o Influence of the AWGN noise to the signal space concept

where w(t) is zero-mean AWGN with PSD N0/2.o After the correlator at the receiver, we obtain:

n Equivalently,n Notably, there is no “information loss”

by the signal space representation.

)()()( twtstx i +=

.jijj wsx +=

)(),()(),()(),( tftwtftstftx jjij += )(tx

)(tfN

)(1 tf

Nx

××,

××,

1x



o Statistics of {wj}

o Since {sij} is deterministic, the distribution of x is a mean-shift of the distribution of w.

o Observe that w is Gaussian distributed because w(t) is AWGN. The distribution of w can, therefore, be determined by its mean vector and covariance matrix.

úúú

û

ù

êêê

ë

é+úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

NiN

i

N w

w

s

s

x

x!!!111



o Mean

o Covariance

[ ] 0)()]([)()(][00

=== òòT

j

T

jj dttftwEdttftwEwE

( )( )[ ]

ij

T

ji

T T

ji

T T

ji

T

j

T

iji

NdttftfN

dsdttfsftsN

dsdttfsftwswE

dttftwdssfswEwwE

d

d

2)()(

2

)()()(2

)()()]()([

)()()()(][

0

0

0

0 0

0

0 0

00

==

-=

=

=

ò

ò ò

ò òòò



o As a result, [w1, w2, …, wN] are zero-mean i.i.d. Gaussian distributed with variance N0/2.

o This shows that x is independent Gaussian distributed with common variance N0/2 and mean vector si = [si1, si2, …, siN]. Equivalently,

Õ=

úû

ùêë

é--=

N

jijji sx

NNf

1

2

00

)(1

exp1

)|(p

sx



o Remainding term in noisen It is possible that

n However, it can be shown that (as an error term) w’(t) is orthogonal to si(t) for 1 £ i £ M. Hence, w’(t) will not affect the decision error rate on message i.

0)()()('1

¹×-= å=

N

iii tfwtwtw

1.y probabilit with0)(),(' =tstw i

o An equivalent signal-space channel model

o The best decision function d( ) that minimizes the decision error is:

n This is the maximum a posteriori probability (MAP) decision rule.


5.4 Likelihood Functions

}|{maxarg1 allfor }|{}|{ if ,)(

},...,{ 1

xxxx

mPMkmPmPmd

Mmmm

kii

Î=

££³=


5.4 Likelihood Functionso With equal prior probabilities, d(x) = arg max

m∈{m1,...,mM }P{m | x}

= argmax P{m1 | x},P{m2 | x},...,P{mM | x}{ }

= argmax P{m1 | x} f (x)P(m1)

, P{m2 | x} f (x)P(m2 )

,..., P{mM | x} f (x)P(mM )

⎧⎨⎩

⎫⎬⎭

= argmax P{m1 | x} f (x)1 /M

, P{m2 | x} f (x)1 /M

,..., P{mM | x} f (x)1 /M

⎧⎨⎩

⎫⎬⎭

= argmax f (x |m1), f (x |m2 ),..., f (x |mM ){ }f(x|mi) is named the likelihood function given mi is transmitted. Hence, the above rule is named the maximum-likelihood (ML) decision rule.


5.4 Likelihood Functions

o MAP rule = ML rule, if equal prior probability is assumed.o In practice, it is more convenient to work on the log-

likelihood functions, defined by

o Why log-likelihood functions are more convenient? The decision function becomes “sum of (squared) Euclidean distances” in AWGN channel.

{ }{ })|(log),...,|(log),|(logmaxarg

)|(),...,|(),|(maxarg)(

21

21

M

M

mfmfmfmfmfmfd

xxxxxxx

==

d(x) = argmax1≤i≤M

log f (x |mi ) = argmax1≤i≤M

log f (x | si )

= argmax1≤i≤M

log 1πN0

exp −1N0

(x j − sij )2

⎡

⎣⎢

⎤

⎦⎥

j=1

N

∏

= argmax1≤i≤M

−12

logπN0 −1N0

(x j − sij )2

⎛

⎝⎜

⎞

⎠⎟

j=1

N

∑

= argmin1≤i≤M

(x j − sij )2

j=1

N

∑

= argmin1≤i≤M

|| x − si ||2 ( = argmin1≤i≤M

|| x − si ||)


Upon the reception of received signal point x, find the signal point si that is closest in Euclidean distance to x.


5.5 Coherent Detection of Signals in Noise: Maximum Likelihood Decoding

decision region for s1




o Signal constellationn The set of M signal points in the signal space

o Example. Signal constellation for 2B1Q code



o Decision regions for N = 2 and M = 4


5.5 Coherent Detection of Signals in Noise: Maximum Likelihood Decodingo Usually, s1, s2, …, sM are named the message points.o The received signal point x wanders about the transmitted

message point in a Gaussian-distributed random fashion.



o Constant-energy signal constellationn In this case, the ML decision rule can be reduced to an

inner-product.

( )( )

constant. is if ,,maxarg

,2minarg

||||,2||||minarg

||||minarg)(

1

1

22

1

2

1

iiMi

iiMi

iiMi

iMi

E

E

d

sx

sx

ssxx

sxx

££

££

££

££

=

+-=

+-=

-=


5.6 Correlation Receiver

o If signals do not have equal energy, we can use

to implement the ML rule.n The receiver is coherent because the receiver requires to

be in perfect synchronization with the transmitter (more specifically, the integration must begin at exactly the right time instance).

.21,maxarg)(

1÷øö

çèæ -=

££ iiMiEd sxx

demodulator or detector

decision maker


N product-integrators or

correlators

Correlation receiver

1x

2x

Nx

matched filter decision maker


N product-integrators or

correlators

1x

2x

Nx


5.6 Equivalence of Correlation and Matched Filter Receivers

o The correlator and matched filter can be made equivalent.o Specifically,


5.7 Probability of Symbol Error

o Average probability of symbol error

{ }åò

å

å

å

=

=¹££

=

=

-=

-£--=

=-=

=-=-=

M

i Zi

M

iijijMji

M

iii

M

iiiice

i

dfM

mM

mmdPM

mmdPmPPP

1

1

2

,1

2

1

1

)|(11

ed transmitt||||min|||| Pr11

ed) transmitt|)((11

ed) transmitt|)(()(11

xsx

sxsx

x

x

{ }.||||min||:|| where 2

,1

2jijMji

NiZ sxsxx -£-ÂÎ=

¹££


5.7 Invariance of Probability of Symbol Error

o Probability of symbol error is invariant with respect to basis change (i.e., rotation and translation of the signal space).

o Specifically, the symbol error rate (SER) only depends on the relative “Euclidean distances” between the message points.

{ }å=

¹££-£--=

M

iijijMjie m

MP

1

2

,1

2 ed transmitt||||min|||| Pr11 sxsx



o Specifically, if Q is a reversible transform (matrix), such as rotation, then

{ }{ }2

,1

2

2

,1

2

||||min||:||

||||min||:||

jijMjiN

jijMjiN

sxsxx

sxsxx

QQQQ -£-ÂÎ=

-£-ÂÎ

¹££

¹££



o A pair of signal constellation for illustrating the principle of rotational invariance.



o The invariance in SER for translation can be likewise proved.

n Is the transmission power the same for both constellations?


5.7 Minimum Energy Signals

o Since SER is invariant to rotation and translation, we may rotate and translate the signal constellation to minimize the transmission power without affecting SER.

n Since Q does not change the norm (i.e., transmission power), we only need to determine the right a.

å=

=M

iiig pE

1

2|||| s

minimized. is ||)(||),( that such and Find1

2å=

-=M

iiig pE asaa QQQ



o Determine the optimal a.

( )

2

11

2

1

22

1

2

||||2||||

||||2||||

||||)(

asas

asas

asa

+÷øö

çèæ-=

+-=

-=

åå

å

å

==

=

=

M

iii

TM

iii

M

ii

Tii

M

iiig

pp

p

pE

å=

=ÞM

iiip

1optimal sa

2

11

2optimal ||||)( and åå

==

-=M

iii

M

iiig ppE ssa



o So, subfigure (a) below has minimum average energy.


5.7 Union Bound on Probability of Error

o Union bound

{ }( )! ( )

( )! ( )

{ }å å

å

åå

å

= ¹=

=¹

=¹

=

=¹££

->-£

ïþ

ïýü

ïî

ïíì

->-Ú

Ú->-=

ïþ

ïýü

ïî

ïíì

-£-Ù

Ù-£--÷øö

çèæ=

-£--=

M

i

M

ijjiji

M

ii

Miij

i

M

ii

Miij

iM

i

M

iijijMjie

mM

mM

mMM

mM

P

1 ,1

22

1

22

21

2

1

22

21

2

1

1

2

,1

2

ed transmitt |||||||| Pr1

ed transmitt ||||||||||||||||

Pr1

ed transmitt ||||||||||||||||

Pr111

ed transmitt||||min|||| Pr11

sxsx

sxsx

sxsx

sxsx

sxsx

sxsx

"

"

"

"

)()()( BPAPBAP +£!



( )( )( )ïþ

ïý

ü

ïî

ïí

ì

->-Ú

->-Ú

->-

24

21

23

21

22

21

||||||||||||||||||||||||

sxsx

sxsx

sxsx

{ }222

1 |||||||| sxsx ->- { }232

1 |||||||| sxsx ->- { }242

1 |||||||| sxsx ->-



å å= ¹=

£M

i

M

ijjjie P

MP

1 ,12 ),(1 ss

{ }. mean withddistribute Gaussian is ed, transmitt given Notably,

.ed transmitt |||||||| Pr),( where 222

ii

ijiji

m

mP

sx

sxsxss ->-=

is js w

ji ss -21



Since x = si + w when si was transmitted, we have

Pr��x � si�2 > �x � sj�2

��si transmitted�

= Pr��(si + w) � si�2 > �(si + w) � sj�2


= Pr��w�2 > �w + (si � sj)�2


= Pr��w�2 > �w�2 + �si � sj�2 + 2(si � sj)

T w��si transmitted

�

= Pr

�(si � sj)

T w < �1

2�si � sj�2

��si transmitted

�

= Pr

�n < �1

2�si � sj�2


�

where n � (si � sj)T w.

Chapter 5-53

5.7 Union Bound on Probability of ErrorObserve that w is zero-mean Gaussian distributedwith covariance matrix E[wwT ] = N0

2 I, where I is the identity matrix.

Hence, n � (si � sj)T w is Gaussian distributed with

E[n] = E�(si � sj)

T w�

= (si � sj)T E[w] = (si � sj)

T 0 = 0

This implies that w � n/�si � sj� is Gaussian distributedwith mean zero and variance N0/2.

E[n2] = E�(si � sj)

T w · ((si � sj)T w)T

�

= E�(si � sj)

T wwT (si � sj)�

= (si � sj)T E

�wwT

�(si � sj)

=N0

2(si � sj)

T I(si � sj)

=N0

2�si � sj�2.


5.7 Union Bound on Probability of ErrorAs a result,

Pr

�n < �1

2�si � sj�2


�

= Pr

��si � sj�w < �1

2�si � sj�2


�

= Pr

�w < �1

2�si � sj�


�

= Pr

�w >

1

2�si � sj�


�,

where the last equality is valid because the probability density function of azero-mean Gaussian random variable is symmetric with respect to w = 0 (hence,Pr[w > a] = Pr[w < �a] for any a > 0).



o Hence,n when N = 1,

{ }

.)exp(2erfc(u) where,2

erfc21

|| where,exp1

||21

Pr

ed transmitt |||||||| Pr),(

2

0

2/0

2

0

222

ò

ò

¥

¥

-=÷÷ø

öççè

æ=

-=÷÷ø

öççè

æ-=

þýü

îíì ->=

->-=

u

ij

jiijd

ji

ijiji

dzzNd

ssddvNv

N

ssw

mP

ij

p

p

sxsxss

n For N = 2,

n The same formula is valid for any N.



w

ji ss -21

P2 (si, s j ) = Pr || x − si ||2>|| x − s j ||2 mi transmitted{ }

= Pr w1 >12dij and w2 = don't care

⎧⎨⎩

⎫⎬⎭

, where dij =|| si − s j ||

=1πN0

exp −v2

N0

⎛

⎝⎜

⎞

⎠⎟dv

dij /2

∞

∫

=12

erfcdij

2 N0

⎛

⎝⎜⎜

⎞

⎠⎟⎟, where erfc(u) = 2

πexp(−z2 )dz

u

∞

∫ .



o Consequently, the union bound for symbol error rate is:

o The above bound can be further simplified when additional condition is given.n For example, if the signal constellation is circularly

symmetric in the sense that “{di1, di2, …, diM} is a permutation of {dk1, dk2, …, dkM} for i ¹ k,” then

å åå å= ¹== ¹=

÷÷ø

öççè

æ=£

M

i

M

ijj

ijM

i

M

ijjjie N

dM

PM

P1 ,1 01 ,1

2 2erfc211),(1 ss

å¹=

÷÷ø

öççè

æ£

M

ijj

ije N

dP

,1 02erfc21



o Another simplification of union boundn Define the minimum distance of a signal constellation

as:

Then, by the strict decreasing property of erfc function,

ijjiMjMidd

¹££££=

,1,1min min

÷÷ø

öççè

æ£÷

÷ø

öççè

æ

0

min

0 2erfc

2erfc

Nd

Ndij

÷÷ø

öççè

æ-=÷

÷ø

öççè

æ£÷

÷ø

öççè

æ£Þ å åå å

= ¹== ¹= 0

min

1 ,1 0

min

1 ,1 0 2erfc

21

2erfc211

2erfc211

NdM

Nd

MNd

MP

M

i

M

ijj

M

i

M

ijj

ije



o We may use the bound for erfc function to realize the relation between SER and dmin.

n Conclusion: SER decreases exponentially as the squared minimum distance grows.

608131.0for )exp()(erfc2

>-

£ uuup

.47929.1 if ,4

exp2

12

erfc2

10

2min

0

2min

0

min NdNdM

NdMPe >÷÷

ø

öççè

æ-

-£÷

÷ø

öççè

æ-£Þ

p


5.7 Relation between BER and SER

o The information bits are transmitted in group of log2M bits to form an M-ary symbol.

o This gives the result that a large symbol error rate (SER) may not cause a large bit error rate (BER).n For example, a symbol error (for large M) may be due to

only 1 bit error.n Optimistically, if every symbol error is due to a single bit

error, then (assuming that n symbols are transmitted)

.)(log

SER)(log

SER BER22 MMn

n=

××

= ÷÷ø

öççè

æ³ .

)(logSER

BERgeneral, In2 M



n Pessimistically, if every symbol error causes log2M bit errors, then (assuming that n symbols are transmitted)

n Summary:

.SERlog

SERlog BER2

2 =×

××=

MnMn ( ).SER BERgeneral, In £

SER BERlogSER

2

££M



o If the statistics for “number of bit error patterns that causes one symbol error” is known, we can determine the exact relation between BER and SER.

Mn

PnM

jjj

2

1

1

log

)()(#SER BER

×

×××=

å-

=

bb

where #(bj ) = number of 1's in bj, and bj represents a binary permutation of log2M bit pattern.

Here, a 1’s in bj means a bit error occurs in the corresponding position; hence, the all-zero pattern is excluded because it represents no symbol error.



o Example. If all bit error patterns (including no error pattern) are equally likely, then

chapter 5 signal-space analysisshannon.cm.nctu.edu.tw/comtheory/chap5.pdf5.2 geometric...

Documents