1

SAMPLE COVARIANCE BASED PARAMETER

ESTIMATION FOR DIGITAL COMMUNICATIONS

Javier Villares Piera

Advisor: Gregori Vázquez Grau

Signal Processing for Communications Group

Dept. of Signal Processing and Communications

Technical University of Catalunya (UPC)

2

OUTLINE

1) INTRODUCTION

2) OPTIMAL SECOND-ORDER ESTIMATION

• LARGE-ERROR

• SMALL-ERROR

3) QUADRATIC EXTENDED KALMAN FILTER

4) SOME ASYMPTOTIC RESULTS

5) CONCLUSIONS

3

OUTLINE

1) INTRODUCTION

2) OPTIMAL SECOND-ORDER ESTIMATION

• LARGE-ERROR

• SMALL-ERROR

3) QUADRATIC EXTENDED KALMAN FILTER

4) SOME ASYMPTOTIC RESULTS

5) CONCLUSIONS

4

PROBLEM STATEMENT

( ) y A θ x wMULTIPLICATIVENON-GAUSSIAN

NOISE

ADDITIVE GAUSSIAN NOISE

ESTIMATE FROM KNOWING STATISTICS ON AND THE

PARAMETERIZATION OF THE PROBLEM

θ y ,x w( )A θ

OBSERVATION

PARAMETERS

5

ESTIMATION PERFORMANCE

2

2 2

( )

( ) ( )

( ) ( ) ( )

BIAS E

VAR E BIAS

MSE E BIAS VAR

y

y

y

θ e

θ e θ

θ e θ θ

( ) e θ θ y DEPENDS ON ANDθ yw

ESTIMATION ERROR

SELF-NOISE

MEASUREMENT NOISE

DETERMINISTIC CASE :

/fy y θLIKELIHOOD

x

6

ESTIMATION PERFORMANCE

2 2

2

0

( )

( )

( )

BIAS E BIAS

VAR E VAR

MSE E MSE BIAS VAR

θ

θ

θ

θ

θ

θ

BAYESIAN CASE :

/f fy θy θ θPRIORLIKELIHOOD

( ) e θ θ y DEPENDS ON ANDθ yw

ESTIMATION ERROR

SELF-NOISE

MEASUREMENT NOISE

x

7

CLASSICAL ESTIMATION CRITERIA

1) MMSE:

2) MVU:

3) ML:

2mi n ( ) mi n ( ) ( )MSE BIAS VAR θ θ

θ θ θ

subject to mi n ( ) ( )MSE BIAS

θθ θ 0

argmax ( / )fyθ

y θ

GENERALLY, NOT REALIZABLE !!

ML MVU MMSEsmall-errorsmall-error

OPTIMALITY :

DIFFICULT !!

8

SMALL-ERROR VS. LARGE-ERROR

( )VAR θ

0o θ θ

SMALL-ERRORLARGE-ERROR

SNR

CRB

ML

OBSERVATION LENGTH INCREASES

THRESHOLD

DETERMINISTIC ESTIMATORS

(ML = MVU = MMSE CRB)

BAYESIAN ESTIMATORS

/f fy θy θ θ /fy y θ 0o θ θ

9

ESTIMATION WITH NUISANCE UNKNOWNS

/fy y θ / ,fy y θ x

CONDITIONALLIKELIHOOD

UNCONDITIONALLIKELIHOOD

NUISANCEPARAMETERS Ex

• Low-SNR UML

• GML x GAUSSIAN

• CML x CONTINUOUS,

DETERMINISTIC

?

10

ESTIMATION WITH NUISANCE UNKNOWNS

/fy y θ / ,fy y θ x

CONDITIONALLIKELIHOOD

UNCONDITIONALLIKELIHOOD

NUISANCEPARAMETERS Ex

• CML x CONTINUOUS,

DETERMINISTIC

?

QUADRATIC

• Low-SNR UML

• GML x GAUSSIAN

11

0 5 10 15 20 25 30 35 40 45 5010

-6

10-5

10-4

10-3

10-2

10-1

100

Es/No (dB)

Nor

mal

ized

Tim

ing

Var

ianc

eQUADRATIC ML-BASED ESTIMATORS COMPARISON

MCRB(x known)

Higher-order

Low-SNR UML

CML

GML

12

GAUSSIAN ASSUMPTION IN COMMUNICATIONS

fx x fx x

-1 1 -1 1

BPSK alphabet

(higher-order info)

Gaussian assumption

(mean and variance info)

?

13

OUTLINE

1) INTRODUCTION

2) OPTIMAL SECOND-ORDER ESTIMATION

• LARGE-ERROR

• SMALL-ERROR

3) QUADRATIC EXTENDED KALMAN FILTER

4) SOME ASYMPTOTIC RESULTS

5) CONCLUSIONS

14

SECOND-ORDER ESTIMATOR

vec Hr yy

H θ b M rESTIMATOR

COEFFICIENTS ?

WITH

SAMPLE COVARIANCE VECTOR

15

ESTIMATOR OPTIMIZATION

• OPTIMUM b :2mi nBIAS

b

• OPTIMUM M :

2 subject to 0mi nVAR BIAS M

2 mi n mi n ( )MSE BIAS VAR M M

2 subject to mi n mi nVAR BIASM M

MVU

1)

2)

3) MVMB

MMSE

TRADE-OFF

16

M OPTIMIZATION: GEOMETRIC INTERPRETATION

(min MSE)M

(min VAR)M

(min BIAS2)MMVMB

MMSE

17

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Normalized Frequency Error

Est

imat

or M

ean

Val

ue MINIMUM BIAS

MMSE(EsNo=10dB)

BIAS MINIMIZATION

SIMULATION PARAMETERS

• FREQ. ESTIMATION

• 2 MSK SYMBOLS

• NSS = 2

UNBIASED

Max. Freq. Error = 0.5

Max. Freq. Error = 1

18

VARIANCE ANALYSIS

Tr ( )HVAR E θM Q θ M

( ) ( ) ( )H

E yQ θ r r θ r r θ

WITHCOVARIANCE

MATRIX OF r

FOURTH-ORDER MOMENTS OF y

19

MATRIX Q()

* * *( ) ( ) ( ) ( ) ( ) ( ) ( )H

Q θ R θ R θ A θ A θ K A θ A θ

( ) ( ) ( (vec vec vec ) vec )H H H HE xK xx xx I I I

WITH

4TH ORDER CUMULANTS (KURTOSIS MATRIX)

NON-GAUSSIAN INFORMATION

K 0 IF x GAUSSIAN !!

20

KURTOSIS MATRIX

2 di ag vec( ) K I

WITH

4

22

x

x

E x

E x

TE xx 0

IF x IS

CIRCULAR

1

2

1 38.

1 32. 4TH TO 2ND ORDER RATIO

M-PSK

16-QAM

64-QAM

GAUSSIAN

21

-20 -10 0 10 20 30 40 50 6010

-4

10-3

10-2

10-1

100

101

Es/No (dB)

MS

E

QUADRATIC ESTIMATORS COMPARISON

SIMULATION PARAMETERS

• FREQ. ESTIMATION

• UNIFORM PRIOR (80% Nyq)

• 4 MSK SYMBOLS

• NSS = 2

Self-noise

MMSE

MVMB

min{BIAS2}

Priorvariance

22

4 8 12 16 20 24 28 32 36 40 4448

10-2

10-1

M

MS

E

Gaussian assumption

ASYMPTOTIC ANALYSIS

SIMULATION PARAMETERS

• FREQ. ESTIMATION

• UNIFORM PRIOR (80% Nyq)

• Es/No = 40dB

• MSK modulation

• NSS = 2

(# of samples)

2 0l i m l i mM M

MSE BIAS

MMSE

MVMB

23

OUTLINE

1) INTRODUCTION

2) OPTIMAL SECOND-ORDER ESTIMATION

• LARGE-ERROR

• SMALL-ERROR

3) QUADRATIC EXTENDED KALMAN FILTER

4) SOME ASYMPTOTIC RESULTS

5) CONCLUSIONS

24

( )fθ θ

oθ

DELTA MEASURE

oθ

( )fθ θ

( )fθ θ

oθ

( )fθ θ

NOT INFORMATIVE ( )fθ θ VERY INFORMATIVE

LARGE-ERROR SMALL-ERROR

25

CLOSED-LOOP ESTIMATION AND TRACKING

DISCRIMINATORor DETECTOR

LOOP FILTERny

n oθ θ

SMALL-ERROR

(STEADY-STATE)

nθ

θ

26

BIAS MINIMIZATION (SMALL-ERROR)

oθ

θ

UNBIASED

( )BIAS θ

( )( )

o

o

BIASBIAS

θ θ

θθ 0

θ

27

BEST QUADRATIC UNBIASED ESTIMATOR (BQUE)

subject to ( )

mi n ( ) ( )o

o o

BIASVAR BIAS

Mθ θ

θθ θ 0

θ

11( ) ( ) ( )H Ho o r o rE

θ θ θ θ D Q θ D

AND WE OBTAIN THAT

2nd-ORDER FIM

LOWER BOUND ON THE VARIANCE OF ANY SECOND-ORDER

UNBIASED ESTIMATOR

K

28

FREQUENCY ESTIMATION PROBLEM

• 2REC MODULATION

• M=8 OBSERVATIONS (NSS=2)

• K=12 NUISANCE PARAM.

• 2REC MODULATION

• M=16 OBSERVATIONS (NSS=4)

• K=12 NUISANCE PARAM.

-10 0 10 20 30 40 50 6010

-6

10-5

10-4

10-3

10-2

10-1

100

Es/No (dB)

Var

ianc

e

Low-SNR UML

CML

GML

BQUEMCRB

UCRB

-10 0 10 20 30 40 50 6010

-6

10-5

10-4

10-3

10-2

10-1

100

Es/No (dB)

Var

ianc

e

Low-SNR UML

CML

GML

BQUEMCRB

UCRB

29

10 20 30 40 50 60 7010

-6

10-5

10-4

10-3

10-2

10-1

SNR (dB)

Nor

mal

ized

var

ianc

e

UCRB

QPSK (BQUE)

MCRB

QPSK (GML)

64-QAM (GML&BQUE)

CHANNEL ESTIMATION PROBLEM

SIMULATION PARAMETERS

• CIR LENGTH 3 SYMB

• 100 GAUSSIAN CHANNELS

• ROLL-OFF = 0.35

• NSS = 3

• OBS. TIME = 100 SYMB.

CONSTANT MODULUS

30

0 5 10 15 20 25 30 35 4010

-6

10-5

10-4

10-3

10-2

10-1

Es/No (dB)

Nor

mal

ized

Var

ianc

e

MCRB

Low-SNR ML

CML

GML

UCRB

BQUE

10 15 20 25 30 35 40 45 50 55 60

10-6

10-4

10-2

100

102

Es/No (dB)

Nor

mal

ized

Var

ianc

e

MCRB

UCRB

Low-SNR ML

CML

GML

BQUE

ANGLE-OF-ARRIVAL ESTIMATION PROBLEM

• M-PSK MODULATION

• 4 ANTENNA

• OBS. TIME = 400 SYMB

SEPARATION 10º SEPARATION 1º

• M-PSK MODULATION

• 4 ANTENNA

• OBS. TIME 3000 SYMB

31

OUTLINE

1) INTRODUCTION

2) OPTIMAL SECOND-ORDER ESTIMATION

• LARGE-ERROR

• SMALL-ERROR

3) QUADRATIC EXTENDED KALMAN FILTER

4) SOME ASYMPTOTIC RESULTS

5) CONCLUSIONS

32

KALMAN FILTER MOTIVATION

1

n n n

n n n

y Hθ v

θ Fθ u

CLOSED-LOOP ESTIMATOR

- OPTIMUM IN THE STEADY-STATE (SMALL-ERROR)

KALMAN FILTER

- OPTIMUM IN THE STEADY-STATE (SMALL-ERROR)

- OPTIMUM IN ACQUISITION (LARGE-ERROR)

MEASUREMENT EQUATION

STATE EQUATIONLINEARGAUSSIAN

LINEARGAUSSIAN

MVU

BAYESIAN MMSE

33

KALMAN FILTER FORMULATION

1

n n n n

n n n

y A θ x w

θ f θ u

vec Hn n n n n n r y y h θ v θ

MEASUREMENT EQUATION

STATE EQUATION

ZERO-MEAN NONLINEAR IN

NONLINEAR IN

ZERO-MEAN

PROBLEM

QUADRATIC OBSERVATION

SAMPLE COV.

VECTOR

NONLINEAR

PROBLEMLINEARIZATION

(EKF FORMULATION)

- NON-GAUSSIAN- DEPENDS ON

( )Hn n nE v v Q θ

34

0 5 10 15 20 25 30 35 4010

-6

10-5

10-4

10-3

10-2

10-1

n (time)

MS

E

Gaussian Assumption

ACQUSITION RESULTS

SIMULATION PARAMETERS

• M-PSK MODULATION

• SNR = 40 dB

• 4 ANTENNAS

SEPARATION = 0.2

SEPARATION = 0.4

35

OUTLINE

1) INTRODUCTION

2) OPTIMAL SECOND-ORDER ESTIMATION

• LARGE-ERROR

• SMALL-ERROR

3) QUADRATIC EXTENDED KALMAN FILTER

4) SOME ASYMPTOTIC RESULTS

5) CONCLUSIONS

36

-20 -10 0 10 20 30 40 50 60 7010

-6

10-5

10-4

10-3

10-2

10-1

100

101

102

Es/No (dB)

Nor

mal

ized

Var

ianc

e

BQUE

UCRB

high-SNR asymptote

low-SNR asymptote

GML

-20 -10 0 10 20 30 40 50 60 7010

-7

10-6

10-5

10-4

10-3

10-2

10-1

100

101

102

Es/No (dB)

Nor

mal

ized

Var

ianc

eBQUE

UCRB

high-SNR asymptote

low-SNR asymptote

GML

high-SNR asymptote

LOW AND HIGH SNR STUDY: DOA

• SEPARATION 1º

• M = 4 ANTENNAS

• SMALL-ERROR

• SEPARATION 1º

• M = 4 ANTENNAS

• SMALL-ERROR

16-QAM (MULTILEVEL) M-PSK (CONSTANT MODULUS)

37

-10 0 10 20 30 40 50 6010

-7

10-6

10-5

10-4

10-3

10-2

10-1

100

101

Es/No (dB)

Nor

mal

ized

Var

ianc

e

M=4,K=6

M=8, K=8

M=10, K=9

M=20, K=14

M

-10 0 10 20 30 40 50 6010

-7

10-6

10-5

10-4

10-3

10-2

10-1

100

101

Es/No (dB)

Nor

mal

ized

Tim

ing

Var

ianc

e

M=8, K=8

M=4,K=6

M

M=20, K=14

M=10, K=9

LARGE SAMPLE STUDY: DIGITAL COMMUNICATIONS

• M-PSK

• NSS = 2

• ROLL-OFF = 0.75

FREQUENCY SYNCHRO. TIMING SYNCHRO.

• M-PSK

• NSS = 2

• ROLL-OFF = 0.75

38

-10 0 10 20 30 40 5010

-7

10-6

10-5

10-4

10-3

10-2

10-1

100

Es/No (dB)

Nor

mal

ized

Var

ianc

e

=2 (UCRB)

=1.2

=1.01

=1.001

=1

asymptote for M

0 10 20 30 40 50 60 7010

-8

10-7

10-6

10-5

10-4

10-3

10-2

10-1

Es/No (dB)

Nor

mal

ized

Var

ianc

e =2 (UCRB)

=1.2

=1.05

=1.01

=1.001

=1

asymptote for M

LARGE SAMPLE RESULTS: DOA

• M = 4 ANTENNAS

• SMALL-ERROR

• M = 4 ANTENNAS

• SMALL-ERROR

SEPARATION 10º SEPARATION 1º

39

101

102

10-12

10-11

10-10

10-9

10-8

10-7

10-6

M

Nor

mal

ized

Var

ianc

e

UCRB, GML

Asymptote for M

BQUE

101

102

10-14

10-12

10-10

10-8

10-6

10-4

M

Nor

mal

ized

Var

ianc

e

UCRB, GML

Asymptote for M

BQUE

LARGE SAMPLE RESULTS: DOA

• M-PSK ( = 1)

• EsNo = 60dB

• SMALL-ERROR

SEPARATION 10º SEPARATION 1º

• M-PSK ( = 1)

• EsNo = 60dB

• SMALL-ERROR

40

OUTLINE

1) INTRODUCTION

2) OPTIMAL SECOND-ORDER ESTIMATION

• LARGE-ERROR

• SMALL-ERROR

3) QUADRATIC EXTENDED KALMAN FILTER

4) SOME ASYMPTOTIC RESULTS

5) CONCLUSIONS

41

CONCLUSIONS

1. IN SECOND-ORDER ESTIMATION, THE GAUSSIAN ASSUMPTION

DOES NOT APPLY FOR

• MEDIUM SNR

• HIGH SNR WITH CONSTANT MODULUS NUISANCE UNKNOWNS,

IF THE OBSERVED VECTOR IS SHORT IN THE PARAMETER

DIMENSION (DOA vs. FREQ.)

2. IN THAT CASE, SECOND-ORDER ESTIMATORS CAN EXPLOIT THE

4TH ORDER INFO. ON THE NUISANCE PARAMETERS

KURTOSIS MATRIX K

42

FURTHER RESEARCH

1. IN MULTIUSER ESTIMATION PROBLEMS…

• CONSTANT MODULUS PROPERTY

• STATISTICAL DEPENDENCE IN CODED TRANSMISSIONS

2. ACQUISITION OPTIMIZATION

3. ESTIMATION AND DETECTION THEORY CONNECTION