1 edt lect1
TRANSCRIPT
-
8/3/2019 1 EDT lect1
1/30
1
EC 6307
Estimation and Detection Theory
Monday: 2 -3 PM
Tuesday: 10.15 11.15 AM
Thursday: 11.15 12.15
Friday: 8 9 AM
-
8/3/2019 1 EDT lect1
2/30
What is EC 6307 About?
Goal: Get useful information out of messy data
Strategy: Formulate probabilistic model of datax, which depends on
underlying parameter(s) U
Terminology depends on parameter space:
Detection (simple hypothesis testing):
U {0,1}, i.e. 0= target absent, 1= target present
Classification (multi-hypothesis testing):
U {0,1,,M}, i.e. U {DC-9, 747, F-15, MiG-31}
Estimation
U Rn 2
-
8/3/2019 1 EDT lect1
3/30
Ex: Positron Emission Tomography
Simple, traditional linear DSP-based approach
Raw DataStatistical Estimate
Advanced, estimation-theoretic approach
3
-
8/3/2019 1 EDT lect1
4/30
Tasks of Statistical Signal Processing
1. Create statistical model for measured data
2. Find fundamental limitations on our ability to perform
inference on the data
Cramr-Rao bound, Chernov bound, etc.
3. Develop an optimal (or suboptimal) estimator
4. Asymptotic analysis (i.e., assume we have lots and lots of data)
of estimator performance to see if it approaches boundsderived in (2)
5. Do simulations and experiments comparing algorithm
performance to lower bounds and competing algorithms4
-
8/3/2019 1 EDT lect1
5/30
First engineering textbook on the topic
Some homework problems are masters thesis topics!!!
Unfortunately, showing its age
All emphasis on continuous time problems, as might be
implemented with analog computers
Ex: Discrete-time Kalman filtering relegated to a homework
problem!!!
Detection, Estimation, and Modulation Theory,
Part I, Harry L. Van Trees, 1968
5
-
8/3/2019 1 EDT lect1
6/30
excellent tutorial and research reference book on estimation theory.
The theory is illustrated with very concrete examples; the examples
give an under-the-hood insight into the solution of some common
estimation problems in signal processing.If you're a statistician, you
might not like this book. If you're an engineer, you will like it.
Amazon review
The theory is explained well and motivated, but what makes the book
great are the examples. There are many worked examples and they are
chosen to make things very clear. Amazon review
Fundamentals of Statistical Signal Processing,Volume 1:
Estimation Theory, Steven Kay, 1993
6
-
8/3/2019 1 EDT lect1
7/30
Pedagogical Question:
What to do first?
View 1: Detection first, then Estimation (Van Trees et. al)
Detection Theory is easier; introduces concepts used in
Estimation Theory in a simple context
View 2: Estimation first, then Detection (Kay)
Detection Theory just a special case of estimation theory
Detection problems with unknown parameters are easier
to think about if youve already seen estimation theory
View 1 seems more common, but well take View 2 7
-
8/3/2019 1 EDT lect1
8/30
8
Read Ch 1 & 2 in :Fundamentals of Statistical Signal
Processing,Volume 1: Estimation Theory by Steven Kay
for the Mondays class
-
8/3/2019 1 EDT lect1
9/30
9
Introduction to Estimation
-
8/3/2019 1 EDT lect1
10/30
An Example Estimation Problem: DSB Receiver
10
-
8/3/2019 1 EDT lect1
11/30
11
Discrete-Time Estimation Problem
These days, we almost always work with samples of the
observed signal (signal plus noise)
Our Thought Model: Each time you observe x[n] it
contains same s[n] but different realization of noise
w[n], so the estimate is different each time.
-
8/3/2019 1 EDT lect1
12/30
12
Our Job: Given finite data setx[0], x[1], x[N-1],
find estimator functions that map data into estimates
-
8/3/2019 1 EDT lect1
13/30
13
PDF of Estimate
Since estimates are RVs, we describe them with a PDF
-
8/3/2019 1 EDT lect1
14/30
14
General Mathematical Statement of EstimationProblem:
What captures all the statistical information needed for
an estimation problem ?
-
8/3/2019 1 EDT lect1
15/30
15
Need the N-dimensional PDF of the data, parameterized by
-
8/3/2019 1 EDT lect1
16/30
16
Ex. Estimating a DC Level in Zero Mean AWGN
Consider a single data point is observed
-
8/3/2019 1 EDT lect1
17/30
17
Soin this case, the parameterization changes the data
PDF would mean
-
8/3/2019 1 EDT lect1
18/30
18
Ex. Modeling Data with Linear Trend
We propose signal and noise models as:
Signal Model: Linear TrendNoise Model: AWGNwith zero mean
-
8/3/2019 1 EDT lect1
19/30
19
Typical Assumptions for Noise Model
W and G is always easiest to analyze
Usually assumed unless you have reason to believe
otherwise
Whiteness is usually the first assumption removed
Gaussian is less often removed due to CLT
Zero Mean is a nearly universal assumption
Most practical cases have zero mean
-
8/3/2019 1 EDT lect1
20/30
20
Variance of noise doesnt always have to be known to
make an estimate
BUT,must know to assess expected goodness of the
estimate
Usually perform goodness analysis as a function of
noise variance (or SNR= Signal-to-Noise Ratio)
Noise variance sets the SNR level of the problem
-
8/3/2019 1 EDT lect1
21/30
21
Classical vs. Bayesian Estimation Approaches
If we view (parameter to estimate) as Non-Random
Classical Estimation
Provides no way to include a priori information about
If we view (parameter to estimate) as Random
Bayesian EstimationAllows use of some a priori PDF on
-
8/3/2019 1 EDT lect1
22/30
22
The first part of the course: Classical Methods
Minimum Variance,Maximum Likelihood, LeastSquares
Second part of the course: Bayesian Methods
MMSE, MAP,Wiener filter,Kalman Filter
Third & fourth parts of the course: Detection theory
-
8/3/2019 1 EDT lect1
23/30
23
Assessing Estimator Performance
Can do this only when the value of is known:
- Theoretical Analysis, Simulations, Field Tests, etc.
Thus it has a PDF of its own and that PDF completely
displays the quality of the estimate.
Often, just capture quality through mean and variance
of the estimator
-
8/3/2019 1 EDT lect1
24/30
24
-
8/3/2019 1 EDT lect1
25/30
25
Equivalent View of Assessing Performance
Completely describe estimator quality with error PDF:
p(e)Desire:
-
8/3/2019 1 EDT lect1
26/30
26
Example: DC Level in AWGN
Model:
PDF of an individual data sample
-
8/3/2019 1 EDT lect1
27/30
27
Uncorrelated Gaussian RVs are Independent
so the joint PDF is the product of the individual PDFs:
-
8/3/2019 1 EDT lect1
28/30
28
-
8/3/2019 1 EDT lect1
29/30
29
-
8/3/2019 1 EDT lect1
30/30
30
Theoretical Analysis vs. Simulations
Ideally wed like to be always be able to theoretically
analyze the problem to find the bias and variance of the
estimator
Theoretical results show how performance depends on
the problem specifications
But sometimes we make use of simulations
to verify that our theoretical analysis is correct
sometimes cant find theoretical results