statistical digital signal processing

8/20/2019 Statistical Digital Signal Processing

1/31

Statistical

SignalProcessing

ECE 5615 Lecture Notes

Spring 2015

© 2007–2015Mark A. Wickert

Equalizer

+1

-1

DistortedInput

EqualizedInput

EstimatedData

x[n]

e[n]

y[n]

-

+

d [n]


2/31


3/31

Chapter 1Course Introduction/Overview

Contents1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . 1-3

1.2 Where are we in the Comm/DSP Curriculum? . . . . 1-4

1.3 Instructor Policies . . . . . . . . . . . . . . . . . . . . 1-6

1.4 The Role of Computer Analysis/Simulation Tools . . . 1-7

1.5 Required Background . . . . . . . . . . . . . . . . . . 1-8

1.6 Statistical Signal Processing? . . . . . . . . . . . . . . 1-9

1.7 Course Syllabus . . . . . . . . . . . . . . . . . . . . . 1-10

1.8 Mathematical Models . . . . . . . . . . . . . . . . . . 1-11

1.9 Engineering Applications . . . . . . . . . . . . . . . . 1-13

1.10 Random Signals and Statistical Signal Processing in

Practice . . . . . . . . . . . . . . . . . . . . . . . . . . 1-14

1-1


4/31

CHAPTER 1. COURSE INTRODUCTION/OVERVIEW

.

1-2 ECE 5615/4615 Statistical Signal Processing


5/31

1.1. INTRODUCTION

1.1 Introduction

Course perspective

Course syllabus

Instructor policies

Software tools for this course

Required background

Statistical signal processing overview

ECE 5615/4615 Statistical Signal Processing 1-3


6/31


7/31

1.3. INSTRUCTOR POLICIES

1.3 Instructor Policies

Working homework problems will be a very important aspect

of this course Each student is to his/her own work and be diligent in keeping

up with problems assignments

Homework papers are due at the start of class

If work travel keeps you from attending class on some evening,

please inform me ahead of time so I can plan accordingly, andyou can make arrangements for turning in papers

The course web site

will serve as an information source between weekly class meet-

ings Please check the web site updated course notes, assignments,

hints pages, and other important course news; particularly ondays when weather may result in a late afternoon closing of thecampus

Grading is done on a straight 90, 80, 70, ... scale with curving

below these thresholds if needed

Homework solutions will be placed on the course Web site inPDF format with security password required; hints pages mayalso be provided



8/31


1.4 The Role of Computer Analysis/

Simulation Tools

In working homework problems pencil and paper type solu-tions are mostly all that is needed

– It may be that problems will be worked at the board bystudents

– In any case pencil and paper solutions are still required tobe turned in later

Occasionally an analytical expression may need to be plotted,here a visualization tool such as Python (via IPython consoleor notebook), MATLAB or Mathematica will be very helpful

Simple simulations can be useful in enhancing your under-standing of mathematical concepts

The use of Python for computer work is encouraged since it isfast and efficient at evaluating mathematical models and run-ning Monte-Carlo system simulations

There will be one or more Python or MATLAB based simula-tion projects, and the Hayes text supports this with MATLABbased exercises at the end of each chapter

– I intend to rework MATLAB examples in Python, includ-ing converting code from the text to freely available Python



9/31

1.5. REQUIRED BACKGROUND

1.5 Required Background

Undergraduate probability and random variables (ECE 3610 or

similar) Linear systems theory

– A course in discrete-time linear systems such as, ECE5650 Modern DSP, is highly recommend (a review is pro-vided in Chapter 2)

Familiarity with linear algebra and matrix theory, as matrix no-tation will be used throughout the course (a review is providedin Chapter 2)

A desire to use the IPython Notebook, but falling back to MATLABor Mathematica if needed. Homework problems will requireits use of some vector/matrix computational tool, and a rea-sonable degree of proficiency will allow your work to proceed

quickly

A desire to dig in!



10/31


1.6 Statistical Signal Processing?

The author points out that the text title is not unique, in fact

A Second Course in Discrete-Time Signal Processing is alsoappropriate

The Hayes text covers:

– Review of discrete-time signal processing and matrix the-ory for statistical signal processing

– Discrete-time random processes

– Signal modeling

– The Levinson and Related Recursions

– Wiener and Kalman filtering

– Spectrum estimation

– Adaptive filters

The intent of this course is not entirely aligned with the texttopics, as this course is also attempting to fill the void leftby Random Signals, Detection and Extraction of Signals from

Noise, and Estimation Theory and Adaptive Filtering

A second edition to the classic detection and estimation theorytext by Van Trees is another optional text for 2015; This is the

text I first learned from

Two Steven Kay books on detection and estimation are nowoptional texts, and may take the place of the Hayes book in thefuture



11/31

1.7. COURSE SYLLABUS

1.7 Course Syllabus

ECE 5615/4615

Statistical Signal ProcessingSpring Semester 2015

Instructor: Dr. Mark Wickert Office: EN292 Phone: [email protected] Fax: 255-3589http://www.eas.uccs.edu/wickert/ece5615/

Office Hrs: Wednesday 10:40–11:45 AM, other times by appointment.

RequiredTexts:

Monson H. Hayes, Statistical Digital Signal Processing and Modeling, John

Wiley, 1996.

OptionalTexts:

H.L. Van Trees, K.L. Bell, and Z. Tian, Detection, Estimation, and Modulation

Theory, Part I , 2nd. ed., Wiley, 2013. Steven Kay, Fundamentals of Statistical

Signal Processing, Vol I: Estimation Theory, Vol II: Detection Theory, Prentice

Hall, 1993/1998. ISBN-978-0133457117/978-0135041352

OptionalSoftware:

Scientific Python (PyLab) via the IPython command line or notebook (http://

ipython.org/ install.html). IPython Notebook is highly recommended. A Linux

Virtual machine is available with all the needed tools. The ECE PC Lab also

has IPython and the notebook installed. If needed the full version of MATLAB

for windows (release 2014b) is also in the lab.

Grading: 1.) Graded homework assignments and computer projects worth 55%.2.) Midterm exam worth 20%.

3.) Final exam worth 25%.

Topics Text Sections

1. Introduction and course overview Notes ch 1

2. Background: discrete-time signal processing, linear algebra 2.1–2.4 (N ch2)

3. Random variables & discrete-time random processes 3.1–3.7 (N ch3)

4. Classical detection and estimation theory N ch4

(optional texts)

5. Signal modeling 4.1–4.8 (N ch5)

6. The Levinson recursion 5.1–5.5 (N ch6)

7. Optimal filters including the Kalman filter 7.1–7.5 (N ch8)

8. Spectrum estimation 8.1–8.8 (N ch9)

9. Adaptive filtering 9.1–9.4

(N ch10)



12/31


1.8 Mathematical Models

Mathematical models serve as tools in the analysis and design

of complex systems

A mathematical model is used to represent, in an approximateway, a physical process or system where measurable quantitiesare involved

Typically a computer program is written to evaluate the math-ematical model of the system and plot performance curves

– The model can more rapidly answer questions about sys-tem performance than building expensive hardware pro-totypes

Mathematical models may be developed with differing degreesof fidelity

A system prototype is ultimately needed, but a computer sim-ulation model may be the first step in this process

A computer simulation model tries to accurately represent allrelevant aspects of the system under study

Digital signal processing (DSP) often plays an important rolein the implementation of the simulation model

If the system being simulated is to be DSP based itself, the sim-ulation model may share code with the actual hardware proto-type



13/31

1.8. MATHEMATICAL MODELS

The mathematical model may employ both deterministic andrandom signal models

!"#$%&'()*+,"(-)./.

0)1/2) 45,)#/$)2( ("6).( *+,"(-)./.

7-+./8'&"# 9/$%&'(/"2 "17#"8)..:9+.()$

;"#))$)2(?

=&& =.,)8(."1 @2()#).(

@2A).(/>'()


14/31


1.9 Engineering Applications

Communications, Computer networks, Decision theory and decision

making, Estimation and filtering, Information processing, Power en-gineering, Quality control, Reliability, Signal detection, Signal anddata processing, Stochastic systems, and others.

Relation to Other Subjects2

!"#$%& ()*#"+,"#$ (-,./&,

01%2"2)+).-

3,.)&".)%#"#$ 4)+./1)#*

()*#"+01%5/,,)#*

!/+)"2)+).-

6/5),)%#78/%1-

9"&/78/%1-

:)#/"1(-,./&,

;%&&


15/31

1.10. RANDOM SIGNALS AND STATISTICAL SIGNAL PROCESSING IN PRACTICE

1.10 Random Signals and Statistical Sig-

nal Processing in Practice

A typical application of random signals concepts involves oneor more of the following:

– Probability

– Random variables

– Random (stochastic) processes

Example 1.1: Modeling with Probability

Consider a digital communication system with a binary sym-metric channel and a coder and decoder

!

"

!

"

#$%&' (&'%&'

" ! !

" ! !

!

!

!"#$% &'()%*!+)(($, -$."#$%

!"#$%&'($##)*+,-)*

',../#"0$1",#

2&31).

4"1(

5%%,%

',#1%,*

&'()%*

/(0"%1)2'"(

3$.$'4$#

/(0"%1)2'"(

! )**+* ,*+-.-/0/'1"

A data link with error correction



16/31


The channel introduces bit errors with probability P e.bit/ D

A simple code scheme to combat channel errors is to repetitioncode the input bits by say sending each bit three times

The decoder then decides which bit was sent by using a major-ity vote decision rule

The system can tolerate one channel bit error without the de-coder making an error

The probability of a symbol error is given by

P e.symbol/ D P .2 bit errors/ C P .3 bit errors/

Assuming bit errors are statistically independent we can write

P .2 bit errors/ D .1 / C .1 /

C .1 / D 32.1 /

P .3 errors/ D D 3

The symbol error probability is thus

P e.symbol/ D 32 23

Suppose P e.bit/ D D 103, then P e.symbol/ D 2:998

106

The error probability is reduced by three orders of magnitude,but the coding reduces the throughput by a factor of three



17/31


Example 1.2: Separate Queues vs A Common Queue

A well known queuing theory result3 is that multiple servers, with

a common queue for all servers, gives better performance than mul-tiple servers each having their own queue. It is interesting to seeprobability theory in action modeling a scenario we all deal with inour lives.

!

"

m

.

.

.

#$%&$%'

()$*+$+$

,-./0/)1 3/)$45$6.%0/)17+'089$%'

:.0$

!

T s

! ;&$%.1$ #$%&/


18/31


Common Queue Analysis

The number of servers is defined to be m, the mean customerarrival rate is per unit of time, and the mean customer servicetime is T s units of time

In the single queue case we let u D T s D traffic intensity

Let D u=m D server utilization

For stability we must have u < m and < 1

As a customer we are usually interested in the average time inthe queue, which is defined as the waiting time plus the servicetime (Tanner)

T CQ

Q D T w C T s D Ec.m; u/T s

m.1 / C T s

D

Ec.m; u/ C m.1 /

T s

m.1 /

where

Ec.m; u/ D um=mŠ

um=mŠ C .1 /Pm1

kD0 uk=kŠ

is known as the Erlang-C formula

To keep this problem in terms of normalized time units, we

will plot T Q=T s versus the traffic intensity u D T s

The normalized queuing time is

T CQ

Q

T sD

Ec.m; u/ C .m u/

m u D

Ec.m; u/

m u C 1



19/31


Separate Queue Analysis

Since the customers randomly choose a queue, arrival rate into

each queue is just =m

The server utilization is D .=m/ T s which is the same asthe single queue case

The average queuing time is (Tanner)

T SQ

Q D T s

1

D T s

1 T sm

D mT s

m T s

The normalized queuing time is

T SQ

Q

T sD

m

m T sD

m

m u

Create the Erlang-C function in MATLAB:

We will plot T Q=T s versus u D T s for m D 2 and 4



20/31


The plots are shown below:

! !"# !"$ !"% !"& ' '"# '"$ '"% '"& #!

'

#

(

$

)

%

*

&

T Q

T s

!!!!!!!

Traffic Intensity!T s

"

Separate

Common

m , #

Queuing time of common queue and separate queues for m D 2 servers



21/31


! !"# $ $"# % %"# & &"# '!

$

%

&

'

#

(

)

*

T Q

T s

!!!!!!!

Traffic Intensity !T s

"

Separate

Common

m , '

Queuing time of common queue and separate queues for m D 4 servers

Example 1.3: Power Spectrum Estimation

The second generation wireless system Global System for Mo-bile Communications (GSM), uses the Gaussian minimum shift-keying (GMSK) modulation scheme

xc.t/ Dp

2P c cos

2f 0t C 2f d

1XnD1

ang.t nT b/

where

g.t/ D 1

2

erf

r 2

ln 2BT b

t

T b

1

2

C erf

r 2

ln 2BT b

t

T bC

1

2



22/31


The GMSK shaping factor is BT b D 0:3 and the bit rate isRb D 1=T b D 270:833 kbps

We can model the baseband GSM signal as a complex randomprocess

Suppose we would like to obtain the fraction of GSM signalpower contained in an RF bandwidth of B Hz centered aboutthe carrier frequency

There is no closed form expression for the power spectrum of a

GMSK signal, but a simulation, constructed in MATLAB, canbe used to produce a complex baseband version of the GSMsignal

!"#$%#&'( *& +,-

. / # ' 0 " 1 2 3 # & 4 * 0 ( 5 6 7 8

GSM Baseband Power Spectrum

B

P fraction

9:;; 9;; ; >;; =;; ;

=;



23/31


Using averaged periodogram spectral estimation we can esti-mate S GMSK.f / and then find the fractional power in any RFbandwidth, B , centered on the carrier

P fraction D

R B=2B=2

S GMSK.f / df R 11

S GMSK.f / df

– The integrals become finite sums in the MATLAB calcu-lation

! "! #!! #"! $!! $"! %!!!

!&$

!&'

!&(

!&)

#

B 200 kHz! 95.6%!

B 100 kHz! 67.8%!

B 50 kHz! 38.0%!

*+ -./012034 2/ 567

+ 8 . 9 3 2 : / . ; < : 1 = 8

GSM Power Containment vs. RF Bandwidth

Fractional GSM signal power in a centered B Hz RF bandwidth

An expected result is that most of the signal power (95%) iscontained in a 200 kHz bandwidth, since the GSM channelspacing is 200 kHz



24/31


Example 1.4: A Simple Binary Detection Problem

Signal x is measured as noise alone, or noise plus signal

x D

(n; only noise for hypothesis H 0V C n; noise + signal for hypothesis H 1

We model x as a random variable with a probability density function dependent upon which hypothesis is present

0 V V 1 ! 1 V 1"1 !

X

p x X H 0! " p x X H 1! "

V T

!

"#$

%&'$()*+,+-.*/01,$

We decide that the hypothesis H 1 is present if x > V T , whereV

T is the so-called decision threshold

The probability of detection is given by

P D D Pr.x > V T jH 1/ DZ 1

V T

px.X jH 1/ dX

0 V V 1 ! 1 V 1"1 !

X

p x

X H 1

! "

V T

P D

Pr x vT

H 1

#! "#

Area corresponding to P D



25/31


We may choose V T such that the probability of false alarm,defined as

P F D Pr.x < V T jH 0/ DZ 1

V T px.X jH 0/ dX

is some desired value (typically small)

0 V V 1 ! 1 V 1"1 !

X

p x

X H 0

! "

V T

P F

Pr x vT

H 0

#! "#

Area corresponding to P F

Example 1.5: A Waveform Estimation Theory Problem

Consider a received phase modulated signal of the form

r.t/ D s.t/ C n.t/ D Ac cos.2f ct C .t// C n.t/

where .t/ D kpm.t/, kp is the modulator phase deviationconstant, m.t/ the modulation, and n.t/ is additive white Gaus-sian noise

The signal we wish to estimate is m.t/, information phasemodulated onto the carrier



26/31


We may choose to develop an estimation procedure that ob-tains Om.t/, the estimate of m.t/ such that the mean square erroris minimized, i.e.,

MSE D Efjm.t/ Om.t/j2g

Another approach is maximum likelihood estimation

A practical implementation of a maximum likelihood basedscheme is a phase-locked loop (PLL)

As a specific example here we consider the MATLAB Simulinkblock diagram shown below:

!"#$%&

!"#$%

'%

()

'%*+,()*- /#0#)$+%1

'*23#)45)6%7&

'*23#)

45)6%7

85+9%

:%2%7*/#7

8;

8;

;#35+*/#7 5

25)@AB

3%2@AB

=C9"7%/%;#35+#

(2/%-7*/#7

&,A,&

/9@&B@AB

=C9"7%/% (2/%-7*/#7

D@ E B

8F

()@5B

0F%"G !C-2*+ H//7C65/%9

&

Complex baseband Simulink block diagram

Results from the simulation using a squarewave input for twodifferent signal-to-noise power ratios (20 dB and 0 dB) areshown below



27/31


! " # $ % & ' ( ) * "!

!

!+&

"

! " # $ % & ' ( ) * "!

!

!+&

"

! " # $ % & ' ( ) * "!

!

!+&

"

,-./

0 1 2 3 4 m 6 t 7

8 9 4 - . : 4 / : 4 # ! ; <

8 9 4 - . : 4 / : 4 ! ; <

Input/output waveforms

Example 1.6: Nonrandom Parameter in White Gaussian Noise

We are given observations

ri D A C ni ; i D 1 ; : : : ; N

with the ni iid of the form N.0; 2n /



28/31


The joint pdf is

prja.RjA/ D

N

Yi D11

p 2 2n exp.Ri A/

2

2 2n

The log likelihood function is

ln prja.RjA/ç D N

2 ln

1

2 2n

1

2 2n

N XiD1

.Ri A/2

Solve the likelihood equation

@ lnŒprja.RjA/ç

@A

ˇ̌̌ˇ

AD Oaml

D 1

2n

N Xi D1

.Ri A/

ˇ̌̌ˇ

AD Oaml

D 0

H) Oaml D 1

N

N

Xi D1Ri

Check the bias:

Ef Oaml.r/ Ag D E

1

N

N Xi D1

ri A

D 1

N

N

Xi D1Efri g„ƒ‚…A

A D 0

No Bias!



29/31


Example 1.7: Sat-Com Adaptive Equalizer

Wideband satellite communication channels are subject to bothlinear and non-linear distortion

Uplink

Channel

Transmitter

Transponder

Receiver

PSK

Mod

HPA

(TWTA)

HPA

(TWTA)

Modulation

Impairments

Bandpass

Filtering

Bandpass

Filtering

Bandpass

Filtering

PSK Demod

(bit true with

full synch)

Adaptive

Equalizer

Other Signals

M o d .

M o d .

WGNNoise

(off)

Data

Source

Recovered

Data

Downlink

Channel

WGN

Noise

(on)

Other

Signals

M o d .

Other

Signals

Spurious PM Incidental AM

Clock jitter

IQ amplitude imbalance IQ phase imbalance

Waveform asymmetryand rise/fall time

Phase noise Spurious PM Incidental AM

Spurious outputs

Phase noise Spurious PM Incidental AM

Spurious outputs

BPSK QPSK

OQPSK

Wideband Sat-Comm simulation model

An adaptive filter can be used to estimate the channel dis-tortion, for example a technique known as decision feedback equalization



30/31


M1 Tap

ComplexFIR

Re

z-1M

1 Tap

Complex

FIR

Im 2

2

M2 Tap

Real

FIR

M2 Tap

Real

FIR

CM Error/

LMS Update

DD Error/

LMS Update

CM Error/

LMS Update

DD Error/

LMS Update

Tap

Weight

UpdateSoft I/Q outputs

from demod at

sample rate = 2Rs

Recovered

I Data

Recovered

Q Data

Stagger for

OQPSK, omit

for QPSK

+

+

-

-

-

-

+

+ AdaptMode

Decision

Feedback

Decision

Feedback

µCM

, µ

DD

µDF

, γ

An adaptive baseband equalizer4

Since the distortion is both linear (bandlimiting) and nonlin-ear (amplifiers and other interference), the distortion cannot becompletely eliminated

The following two figures show first the modulation 4-phasesignal points with and with out the equalizer, and then the bit

error probability (BEP) versus received energy per bit to noisepower spectral density ratio (Eb=N 0)

4Mark Wickert, Shaheen Samad, and Bryan Butler. “An Adaptive Baseband Equalizer for HighData Rate Bandlimited Channels,Ó Proceedings 2006 International Telemetry Conference, Session5, paper 06–5-03.



31/31


−1.5 −1 −0.5 0 0.5 1 1.5−1.5

−1

−0.5

0

0.5

1

1.5

In−phase

Q u a d r a t u r e

−1.5 −1 −0.5 0 0.5 1 1.5−1.5

−1

−0.5

0

0.5

1

1.5

In−phase

Q u a d r a t u r e

Before Equalization: R b = 300 Mbps After Equalization: R

b = 300 Mbps

OQPSK scatter plots with and without the equalizer

6 8 10 12 14 16 18 20 22 2410

−7

10−6

10−5

10−4

10−3

10−2

Eb/N

0 (dB)

P r o b a b i l i t y o f B i t E r r o r

Theory EQ NO EQ

4.0 dB 8.1 dB

Semi-Analytic Simulation

300 MBPS BER Performance with a 40/0 Equalizer

BEP versus Eb=N 0 in dB

statistical digital signal processing

Documents