statistical signal processing · lecture # 1 signal hierarchy examples of statistical experiments...

ECE 600-03Statistical Signal Processing

Aly A. FaragUniversity of Louisville

Spring 2010www.cvip.uofl.edu

Lecture # 1

Signal Hierarchy Examples of Statistical Experiments Outcome

– Seismic recordings– EEG signal– EMG signal– Stock market readings– Radar return signal– Temperature readings in an area over a period

Processing vs. Analysis– ECE 600 will do mix of processing and analysis

Reading Material

Signal Hierarchy

• Deterministic– Can be represented by a specific equations

Example: y = 5 cos 2π t

• Random (stochastic)– Cannot be described by a specific equation; yet

can be modeled by a certain parametric form

Example: y[n] = A + w[n]; where w[.] is an innovation process and A is a constant value.

Examples of Signals( as Outcomes of Statistical Experiments)

Seismic Signals

may register volcanic eruption, an earthquake and other internal earth activities due natural or artificial stimulus (e.g., oil exploration)

From NOVA

Goal: to predict volcanic eruption, for example

Biomedical SignalsEEG Recordings

Goal: to infer thefunctionality of the brainIn response to a stimulus, for example

EMG Recordings

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1455479/figure/F1/

http://www.emglab.net/emglab/Figures/figures.html

Goal: to infer thefunctionality of a muscle in response to a stimulus, for example

The electrocardiogram (ECG) is a technique of recording bioelectric currents generated by the heart. Clinicians can evaluate the conditions of a patient's heart from the ECG and perform further diagnosis. ECG records are obtained by sampling the bioelectric currents sensed by several electrodes, known as leads. A typical one-cycle ECG tracing is shown in Figure 1.

Figure 1: A typical one-cycle ECG tracingGenerally, the recorded ECG signal is often contaminated by noise and artifacts that can be within the frequency band of interest and manifest with similar characteristics as the ECG signal itself. In order to extract useful information from the noisy ECG signals, you need to process the raw ECG signals

ECG Recordings

http://zone.ni.com/devzone/cda/tut/p/id/6349

Goal: to infer thefunctionality of the heartfrom surface measurements.

DOW JONES STOCK MARKET READINGS

http://www.quote.com/beta/chart.action?s=%24INDU

Goal: to infer the “state of the economy”from the trading activities.

The Meltdown!

New President!

Demographic charts

http://www.swivel.com/charts/2396-Number-of-children-born-to-women-aged-15-to-49-in-various-OECD-countries

Goal: to infer trends in certain behavioral issues such as birth rate within a population.

Speech Signal

http://cnx.org/content/m0049/latest/

Goal: to understand someAspects of the speech signals;e.g., type, automatic reading,Identification, etc.

Radar Detection

0 2 4 6 8 10 12 14-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1Clean Sinusoidal Signal

Time

Am

plit

ude

0 2 4 6 8 10 12 14-1.5

-1

-0.5

0

0.5

1

1.5Noisy Sinusoidal Signal

Time

Am

plit

ude

Sent signalReceived signal

From the delay of the received signal, the location of the target can be estimated

Goal: to infer the location of a targetOr set of targets real-time. Radar signals have various uses in tracking and navigation.

Time Series

A Time Series -- A continuous plot of an experiment(stock market reading in our case)

Processing vs. Analysis

• Processing – conditions the outcome to a desired form. For example remove the DC value or enhance the oscillations (remove noise)

• Analysis – interprets the information content in the signal. For example, deciding that the given signal belongs to a particular individual

• The domains of signal processing and analysis is huge! We have our focus in ECE 600 to do a little bit of both which will give us some tools to better understand outcomes of certain engineering problems.

Elements of Our Approach

• Study statistical models– Overview of probability– Overview of Signal Representations– Overview of the Linear Statistical Model

• Examples of Processing– Weiner and Kalman Filters

• Examples of Modeling– Speech– Biomedical Signals– Stock market readings

• 2D Examples– Random Fields– Image Analysis

Random Variate Generation

• The building block is uniform random variable

• Use the inverse formula to generate other distributions

Example: 1D Gaussian from 1D Uniform using the Box-Muller formula:

y1 = sqrt( - 2 ln(x1) ) cos( 2 pi x2 )

y2 = sqrt( - 2 ln(x1) ) sin( 2 pi x2 )

The polar form of the Box-Muller transformation is both faster and more robust numerically. The algorithmic description of it is:

float x1, x2, w, y1, y2;

do { x1 = 2.0 * ranf() - 1.0; x2 = 2.0 * ranf() - 1.0;

w = x1 * x1 + x2 * x2; }

while ( w >= 1.0 );

w = sqrt( (-2.0 * ln( w ) ) / w );

y1 = x1 * w;

y2 = x2 * w;

http://www.taygeta.com/random/gaussian.html

Random Number Generation

1 & 2 Dimensional Uniform Data Streams using the MATLAB command: rand

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1D 2D

Random Number Generation

1 & 2 Dimensional Uniform Histograms

1D 2D

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.002

0.004

0.006

0.008

0.01

0.012Histogram Plot 1D

Random Variable (x)

Fre

quency o

f O

ccura

nce

00.2

0.40.6

0.81

0

0.5

10

0.002

0.004

0.006

0.008

0.01

0.012

Random Variable (X)

Histogram of 2D Uniform

Random Variable (Y)

Fre

quency o

f O

ccura

nce

Random Number Generation

1 & 2 Dimensional Normal Data Streams using the MATLAB command: rand and the Box-Mueller Method

1D 2D

-4 -3 -2 -1 0 1 2 3 4-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

-4 -3 -2 -1 0 1 2 3 4-5

-4

-3

-2

-1

0

1

2

3

4

Random Number Generation

1 & 2 Dimensional Normal Histograms

1D 2D

-5 -4 -3 -2 -1 0 1 2 3 4 50

0.005

0.01

0.015

0.02

0.025

0.03

0.035Histogram Plot 1D of normal distribution

Random Variable (x)

Fre

quency o

f O

ccura

nce

-5

0

5

-5

0

5-0.1

-0.05

0

0.05

0.1

0.15

0.2

0.25

Random Variable (X)

Histogram Plot 2D of normal distribution

Random Variables andRandom Process

A Random Process is an indexed set of random variables. There are four possibilities based on whether the random variables are continuous/discrete and the index is continuous/discrete. Random processes may be stationary, e.g., white noise and non-stationary, e.g., speech signals.

A Random Variable is a mapping from the sample space of an experiment into the real-line. Evens (subsets of the sample space) are mapped into Borel sets on the real line.

.

.

.

Average for all January months: 1970-2009

Random Variables: X1,X2,X3,XnX1 X2 X3 Xn

Week 1 Week 2 Week 3 Week 4

Illustration of a Random Process from an ensemble of readings over the month of January for 39 years (January 1970 – January 2009)

Readings from a certain day form realizations of a random variable

The collections {X1, X2, X3,…, Xn} form a nth

degree random process

Reading materials

1. G. E. Box and G. M. Jenkins, Time Series Analysis, 2nd

Edition, Holden-Day, CA, 1976.2. D. Brillinger, Time-Series, Expanded Edition,

McGraw-Hill, 1975.3. Thomas Kailath, Lectures on Weiner and Kalman

Filtering, Springer, 1981.4. S. Kay, Statictical Signal Processing, Vol 1., Prentice-

Hall, 1993.5. D. Manolakis, V. Ingle and S. Kogon, Statistical and

Adaptive Signal Processing, McGraw-Hill, 2000.6. J. Lim, Two-dimensional Signals and Image

Processing, Prentice-Hall, 1981.