![Page 1: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/1.jpg)
Theory and Applications of Parametric Estimation Methods
for Sensor Array Signal Processing
Chiao-En ChenDepartment of Electrical Engineering
National Chung Cheng University
1Invited talk at NCNU, Mar 05, 2009
![Page 2: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/2.jpg)
2
Outline
OverviewSelected works in sensor array signal processing– Design and implementation of a prototype system for birds
monitoring and vocalization enhancement– Stochastic maximum likelihood Direction-of-Arrival (DOA)
estimation in the presence of unknown non-uniform noiseConclusions
![Page 3: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/3.jpg)
An Overview on the Sensor Networks
A sensor network consists of a number of nodes (possibly randomly distributed), each with – A number of sensors (e.g., acoustic; seismic; magnetic;
chemical; image; video; temperature; etc.)– Processor (low-powered embedded processor of varying
processing capability)– Radio (low-powered transceiver of varying capability and
range)– Battery (often limited energy and size)– Program controlling the entire network
3
LWIN node
AWAIRS node
Mote node (plus other boards)
![Page 4: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/4.jpg)
An Overview on the Sensor Networks
Explosive interests in sensor networks– U.S. National Science Foundation (NSF) is supporting
many large research projects in this area● NEON (National ecological observatory network): 500 millions
– Study the change of weather over a forest canopy● Earthscope: 200 millions
– Track the faint tremors, measure the crustal deformation● Neptune: 200 millions
– Study the 3 D behavior of the ocean environment● CENS (Center for embedded networked sensing) @UCLA: 40
million– Study the impact of densely embedded sensing for scientific
applications
4
![Page 5: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/5.jpg)
Projects at CENS
5
CENS ecosystems‐bio‐complexity study CENS contaminant trasnport study
CENS marine microorganism study CENS seismic structure response study
![Page 6: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/6.jpg)
CENS Ecosystems-Bio-Complexity Project
6
Acorn woodpecker
James Reserve in San Jacinto Mountain in California
Collaborated with the Department of Ecology & Evolutionary Biology Designed a prototype bird localization systemDeveloped many effective algorithms for detection, estimation, tracking, and classification.
![Page 7: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/7.jpg)
CENS Ecosystems-Bio-Complexity Project
7
Acorn Woodpecker(Melanerpes formicivorus)
![Page 8: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/8.jpg)
8
Overview on the sensor networks Selected works in sensory array signal processing– Design and implementation of a prototype system for birds
monitoring and vocalization enhancement– Stochastic Maximum likelihood Direction-of-Arrival (DOA)
estimation in the presence of unknown non-uniform noiseConclusions
![Page 9: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/9.jpg)
9
A Practical System Design ProblemObjective: Design a prototype system for birds localization and vocalization enhancementConstraints and requirements:– Small number of sensors (microphones) per array(node)– Small number of observations (snapshots)– Source separation & SNR enhancement– Robust in mild multipath environment
Proposed design:– Wideband ML estimator– Uniform Circular Array (UCA)
● Uniform performance over 360 degrees (isotropic)● Relatively larger array aperture
UCA
![Page 10: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/10.jpg)
10
Wideband ML DOA Estimation
Wideband signal model (far-field)– M sources, P sensors, N samples (snapshots)
Using DTFT, we have
where
![Page 11: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/11.jpg)
11
Maximum Likelihood Estimation
Put it into matrix form,
AssumeThe log-likelihood function is then expressed as
Deterministic Maximum Likelihood Estimator
unknown parameters
D depends nonlinearly on the DOAs
Uniform white noise assumption
![Page 12: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/12.jpg)
12
UCA Design
Cramer Rao bound (CRB) analysis:
Derivation of the 3dB beam width
Using Taylor’s series expansion
– The 3-dB beam width=
beam‐pattern
![Page 13: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/13.jpg)
13
UCA DesignA more intuitive CRB expression:
R can not be arbitrarily large– Insufficient spatial sampling
causes spatial aliasing– Introduces grating-lobes in
narrowband beam pattern– Introduces side-lobes in
wideband beam patternTrade-off between accuracyand robustness
4‐element UCA, R=7.07 cm, Woodpecker
4‐element UCA, R=2.83 cm, Woodpecker
![Page 14: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/14.jpg)
14
Simulated Beam Patterns
4‐element UCA, R=6.10 cm, Antthrush
4‐element UCA, R=4.24 cm, Antthrush
4‐element UCA, R=7.07 cm, Woodpecker
8‐element UCA, R=7.07 cm, Woodpecker
![Page 15: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/15.jpg)
15
Suggested Design RulesHaving more microphones– More robust (lower sidelobes)– Better accuracy
– Higher complexity
Rule of thumb:– Maximize R while
constraining the sidelobe height to be at most 20% of the mainlobe height
A table of suggested array apertures for different bird species
![Page 16: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/16.jpg)
16
Signal Separation & EnhancementBeamformer:– Single source:
– Multiple sources:
Same as delay‐and‐sum beamformer and the SRP
The array gain is a function of both the DOAs and the frequency
![Page 17: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/17.jpg)
Sources Separation Experiment
17Beamformer input: Woodpecker +Antthrush Beamformer output: Antthrush
Beamformer output: WoodpeckerML metric: 8‐element UCA, R=0.707cm
(187,59)
(59,187)
(180,60)
(60,180)
![Page 18: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/18.jpg)
18
Localization Experiments
True locationEstimated location
True locationEstimated location
![Page 19: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/19.jpg)
Localization Experiment
Related works:– C.E. Chen , A.M. Ali, H. Wang, S. Asgari, H. Park, R.E. Hudson, K.
Yao, and C.E. Taylor, "Design and testing of robust acoustic arrays for localization and enhancement", in Proceedings. of IPSN, Nashville, Tennessee, April, 2006.
– H. Wang, C.E. Chen, A. Ali, S. Asgari, R. E. Hudson, K. Yao, D. Estrin, and C.E. Taylor, "Acoustic sensor networks for woodpecker localization", in Proc. SPIE on Advanced Signal Processing Algorithms, Architectures, and Implementations, vol 5910, Aug. 2005. 19
True locationEstimated location
![Page 20: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/20.jpg)
20
Overview on the sensor networks Selected works in sensory array signal processing– Design and implementation of a prototype system for birds
monitoring and vocalization enhancement– Stochastic Maximum likelihood Direction-of-Arrival (DOA)
estimation in the presence of unknown non-uniform noiseConclusions
![Page 21: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/21.jpg)
Non-uniform ML estimator
Motivation:– M. Pesavento and A.M. Gershman, “Maximum-likelihood
direction-of-arrival estimation in the presence of unknown nonuniform noise’’, IEEE Trans. Signal Processing, vol. 49, page 1310-1324, July 2001.
Ideas:– For sparsely distributed arrays, the noise at each sensor
can be modeled as uncorrelated but the variances can benon-identical (non-uniform noise environment)
– Uniform MLE blindly treats all the sensors equally– Estimators using general colored noise modeling schemes
neglect the fact that the noise is uncorrelated
21
They are suboptimal in the non-uniform noise case
![Page 22: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/22.jpg)
Non-uniform ML estimator
In Pesavento & Gershman’s paper:– Derived both the deterministic and stochastic non-uniform
CRB– Proposed a deterministic non-uniform ML DOA estimator
based on stepwise concentration – A more difficult problem: the stochastic non-uniform ML
DOA estimator is unsolvedOur work:– Stochastic non-uniform ML DOA estimator
● C.E. Chen and et.al., ``Stochastic Maximum Likelihood DOA Estimation in the Presence of Unknown Nonuniform Noise’’, IEEE Trans. Signal Processing (correspondence), vol. 56, no. 7, pp. 3038-3044, July, 2008
22
![Page 23: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/23.jpg)
Narrowband Array Signal Model
23
where
Stochastic process
Assuming M sources, P sensors, N snapshots
Nonuniform noise
Unknown parameters
be the array observation
be the steering matrix
be the transmitted signal
be the sensor noise
Array Signal Model
![Page 24: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/24.jpg)
Maximum Likelihood DOA Estimation
24
where
Stochastic Maximum Likelihood Estimator
is the sample covariance matrix of y
is the true covariance matrix of y
Can be viewed as a covariance matching problem:We try to find a Ry with a specified structure that best matches S in the ML sense.
![Page 25: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/25.jpg)
Maximum Likelihood DOA Estimation
Stochastic ML Uniform Estimator:
Concentrated stochastic MLE [Stoica & Nehorai, 1995]
25
Analytically concentratedexpression
If
Dimension of the search space:
Dimension of the search space:
![Page 26: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/26.jpg)
Stochastic Narrowband Non-uniformML DOA Estimator
Q1: Can we find an analytically concentrated expression in the case of stochastic model & non-uniform noise?A1: No. All the works in the literature suggest that this might be impossible.Suboptimal algorithms have been proposed:– Approximate ML (AML) method:
● B. Goransson and B. Ottersten, “Direction estimation in partially unknown noise fields,” IEEE Trans. on Signal Processing, September 1999
● High complexity● Uses large sample approximation● Achieves the CRB when N large
– Power domain method:● D. Madurasinghe, “A new DOA estimator in nonuniform noise,”
IEEE Signal Processing Letters, April 2005.● Low complexity ; does not achieve CRB
26
![Page 27: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/27.jpg)
Stochastic Narrowband Non-uniformML DOA Estimator
Q2: Can we do better?A2: Yes!– Stepwise-Concentration + Modified Inverse Iteration
Algorithm● Moderate complexity● Achieves the CRB● Best performance when N is small
27
![Page 28: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/28.jpg)
Stochastic Narrowband Non-uniformML DOA Estimator
Derivation:
28
We obtain an analytical expression of Rx as a function of θ and q
where
![Page 29: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/29.jpg)
Stochastic Narrowband Non-uniformML DOA Estimator
– Substituting back into
29
where
Concentrated log‐likelihood function for θ and q
![Page 30: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/30.jpg)
Implementation Based on Step-wise Concentration
Stepwise-Concentration
30
Concentrateduniform MLE(Stoica, Nehorai)
Numerical procedure to find an improved estimate
![Page 31: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/31.jpg)
Refining on q
General Method:– Given some initial estimates , , and
find an improving direction– Update– Repeat until the algorithm converges
The Newton’s Method: – standard technique for convex optimization problems– The Newton’s direction is found by solving
31
![Page 32: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/32.jpg)
The Newton’s Method
It turns out is not guaranteed to be positive definite.
We are facing a challenging non-convex optimizationproblem !
is not applicable, since it is not guaranteed to be an improving direction !
We propose the Modified Inverse Iteration Algorithm.
32
![Page 33: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/33.jpg)
Modified Inverse Iteration Algorithm
∆q is an improving direction iff
If we choose ∆Q such that
33
>0
then ∆q is an improving direction
Key Step
It turns out that this direction leads to a bound achieving solution for DOA estimation!!
![Page 34: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/34.jpg)
Modified Inverse Iteration Algorithm
After some manipulations, the condition
Note that the Newton’s method has a similar structure
34
where
Modified Inverse Iteration Algorithm
where
![Page 35: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/35.jpg)
Modified Inverse Iteration Algorithm
Note that
When conditions 1. and 2. both hold, then
35
N goes to infinity1.
2.
When condition 1. and 2. both bold, the modified inverse iteration algorithm is essentially the Newton’s method!!
![Page 36: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/36.jpg)
Modified Inverse Iteration Algorithm
In practice,– N is always limited, likely to be small– The initial estimates can never be perfect
As a result, the modified inverse iteration algorithm can be considered as a novel modification of the Newton’s method for the considered non-convex optimization problem.
36
![Page 37: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/37.jpg)
Simulation
37
Number of snapshots (N)
RM
SE
(deg
)
RM
SE
(deg
)SNR(dB)
sto-MLE (uniform)PD method sto MLE (2nd iteration) AML methodsto CRB
sto-MLE (uniform)PD method sto MLE (2nd iteration) AML methodsto CRB
Algorithms Multiple-dimensional searches ComplexityPD One M-D searchAML One M-D searchSto-MLE Two M-D search + one P-D search
![Page 38: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/38.jpg)
Conclusions
In this talk, discussed some of the important issues in the theory and applications of sensor array signal processing– Birds localization and vocalization enhancement
● Design trade-offs (accuracy, robustness, complexity) ● Results from the real-life experiment has been demonstrated
– Stochastic ML DOA estimation in the presence of unknown non-uniform noise● Stepwise concentration & modified inverse iteration algorithm● Achieves the CRB with moderate complexity● Best performance when N is small
38
![Page 39: Theory and Applications of Parametric Estimation Methods for](https://reader034.vdocument.in/reader034/viewer/2022051716/58a02dc61a28abd3308b6b99/html5/thumbnails/39.jpg)
Thank you for your attention
39