theory and applications of parametric estimation methods for

Theory and Applications of Parametric Estimation Methods

for Sensor Array Signal Processing

Chiao-En ChenDepartment of Electrical Engineering

National Chung Cheng University

1Invited talk at NCNU, Mar 05, 2009

2

Outline

OverviewSelected works in sensor array signal processing– Design and implementation of a prototype system for birds

monitoring and vocalization enhancement– Stochastic maximum likelihood Direction-of-Arrival (DOA)

estimation in the presence of unknown non-uniform noiseConclusions

An Overview on the Sensor Networks

A sensor network consists of a number of nodes (possibly randomly distributed), each with – A number of sensors (e.g., acoustic; seismic; magnetic;

chemical; image; video; temperature; etc.)– Processor (low-powered embedded processor of varying

processing capability)– Radio (low-powered transceiver of varying capability and

range)– Battery (often limited energy and size)– Program controlling the entire network

3

LWIN node

AWAIRS node

Mote node (plus other boards)

An Overview on the Sensor Networks

Explosive interests in sensor networks– U.S. National Science Foundation (NSF) is supporting

many large research projects in this area● NEON (National ecological observatory network): 500 millions

– Study the change of weather over a forest canopy● Earthscope: 200 millions

– Track the faint tremors, measure the crustal deformation● Neptune: 200 millions

– Study the 3 D behavior of the ocean environment● CENS (Center for embedded networked sensing) @UCLA: 40

million– Study the impact of densely embedded sensing for scientific

applications

4

Projects at CENS

5

CENS ecosystems‐bio‐complexity study CENS contaminant trasnport study

CENS marine microorganism study CENS seismic structure response study

CENS Ecosystems-Bio-Complexity Project

6

Acorn woodpecker

James Reserve in San Jacinto Mountain in California

Collaborated with the Department of Ecology & Evolutionary Biology Designed a prototype bird localization systemDeveloped many effective algorithms for detection, estimation, tracking, and classification.

CENS Ecosystems-Bio-Complexity Project

7

Acorn Woodpecker(Melanerpes formicivorus)

8

Overview on the sensor networks Selected works in sensory array signal processing– Design and implementation of a prototype system for birds

monitoring and vocalization enhancement– Stochastic Maximum likelihood Direction-of-Arrival (DOA)


9

A Practical System Design ProblemObjective: Design a prototype system for birds localization and vocalization enhancementConstraints and requirements:– Small number of sensors (microphones) per array(node)– Small number of observations (snapshots)– Source separation & SNR enhancement– Robust in mild multipath environment

Proposed design:– Wideband ML estimator– Uniform Circular Array (UCA)

● Uniform performance over 360 degrees (isotropic)● Relatively larger array aperture

UCA

10

Wideband ML DOA Estimation

Wideband signal model (far-field)– M sources, P sensors, N samples (snapshots)

Using DTFT, we have

where

11

Maximum Likelihood Estimation

Put it into matrix form,

AssumeThe log-likelihood function is then expressed as

Deterministic Maximum Likelihood Estimator

unknown parameters

D depends nonlinearly on the DOAs

Uniform white noise assumption

12

UCA Design

Cramer Rao bound (CRB) analysis:

Derivation of the 3dB beam width

Using Taylor’s series expansion

– The 3-dB beam width=

beam‐pattern

13

UCA DesignA more intuitive CRB expression:

R can not be arbitrarily large– Insufficient spatial sampling

causes spatial aliasing– Introduces grating-lobes in

narrowband beam pattern– Introduces side-lobes in

wideband beam patternTrade-off between accuracyand robustness

4‐element UCA, R=7.07 cm, Woodpecker


14

Simulated Beam Patterns

4‐element UCA, R=6.10 cm, Antthrush

4‐element UCA, R=4.24 cm, Antthrush



15

Suggested Design RulesHaving more microphones– More robust (lower sidelobes)– Better accuracy

– Higher complexity

Rule of thumb:– Maximize R while

constraining the sidelobe height to be at most 20% of the mainlobe height

A table of suggested array apertures for different bird species

16

Signal Separation & EnhancementBeamformer:– Single source:

– Multiple sources:

Same as delay‐and‐sum beamformer and the SRP

The array gain is a function of both the DOAs and the frequency

Sources Separation Experiment

17Beamformer input: Woodpecker +Antthrush Beamformer output: Antthrush

Beamformer output: WoodpeckerML metric: 8‐element UCA, R=0.707cm

(187,59)

(59,187)

(180,60)

(60,180)

18

Localization Experiments

True locationEstimated location


Localization Experiment

Related works:– C.E. Chen , A.M. Ali, H. Wang, S. Asgari, H. Park, R.E. Hudson, K.

Yao, and C.E. Taylor, "Design and testing of robust acoustic arrays for localization and enhancement", in Proceedings. of IPSN, Nashville, Tennessee, April, 2006.

– H. Wang, C.E. Chen, A. Ali, S. Asgari, R. E. Hudson, K. Yao, D. Estrin, and C.E. Taylor, "Acoustic sensor networks for woodpecker localization", in Proc. SPIE on Advanced Signal Processing Algorithms, Architectures, and Implementations, vol 5910, Aug. 2005. 19


20

Overview on the sensor networks Selected works in sensory array signal processing– Design and implementation of a prototype system for birds

monitoring and vocalization enhancement– Stochastic Maximum likelihood Direction-of-Arrival (DOA)


Non-uniform ML estimator

Motivation:– M. Pesavento and A.M. Gershman, “Maximum-likelihood

direction-of-arrival estimation in the presence of unknown nonuniform noise’’, IEEE Trans. Signal Processing, vol. 49, page 1310-1324, July 2001.

Ideas:– For sparsely distributed arrays, the noise at each sensor

can be modeled as uncorrelated but the variances can benon-identical (non-uniform noise environment)

– Uniform MLE blindly treats all the sensors equally– Estimators using general colored noise modeling schemes

neglect the fact that the noise is uncorrelated

21

They are suboptimal in the non-uniform noise case

Non-uniform ML estimator

In Pesavento & Gershman’s paper:– Derived both the deterministic and stochastic non-uniform

CRB– Proposed a deterministic non-uniform ML DOA estimator

based on stepwise concentration – A more difficult problem: the stochastic non-uniform ML

DOA estimator is unsolvedOur work:– Stochastic non-uniform ML DOA estimator

● C.E. Chen and et.al., ``Stochastic Maximum Likelihood DOA Estimation in the Presence of Unknown Nonuniform Noise’’, IEEE Trans. Signal Processing (correspondence), vol. 56, no. 7, pp. 3038-3044, July, 2008

22

Narrowband Array Signal Model

23

where

Stochastic process

Assuming M sources, P sensors, N snapshots

Nonuniform noise

Unknown parameters

be the array observation

be the steering matrix

be the transmitted signal

be the sensor noise

Array Signal Model

Maximum Likelihood DOA Estimation

24

where

Stochastic Maximum Likelihood Estimator

is the sample covariance matrix of y

is the true covariance matrix of y

Can be viewed as a covariance matching problem:We try to find a Ry with a specified structure that best matches S in the ML sense.

Maximum Likelihood DOA Estimation

Stochastic ML Uniform Estimator:

Concentrated stochastic MLE [Stoica & Nehorai, 1995]

25

Analytically concentratedexpression

If

Dimension of the search space:

Dimension of the search space:

Stochastic Narrowband Non-uniformML DOA Estimator

Q1: Can we find an analytically concentrated expression in the case of stochastic model & non-uniform noise?A1: No. All the works in the literature suggest that this might be impossible.Suboptimal algorithms have been proposed:– Approximate ML (AML) method:

● B. Goransson and B. Ottersten, “Direction estimation in partially unknown noise fields,” IEEE Trans. on Signal Processing, September 1999

● High complexity● Uses large sample approximation● Achieves the CRB when N large

– Power domain method:● D. Madurasinghe, “A new DOA estimator in nonuniform noise,”

IEEE Signal Processing Letters, April 2005.● Low complexity ; does not achieve CRB

26


Q2: Can we do better?A2: Yes!– Stepwise-Concentration + Modified Inverse Iteration

Algorithm● Moderate complexity● Achieves the CRB● Best performance when N is small

27


Derivation:

28

We obtain an analytical expression of Rx as a function of θ and q

where


– Substituting back into

29

where

Concentrated log‐likelihood function for θ and q

Implementation Based on Step-wise Concentration

Stepwise-Concentration

30

Concentrateduniform MLE(Stoica, Nehorai)

Numerical procedure to find an improved estimate

Refining on q

General Method:– Given some initial estimates , , and

find an improving direction– Update– Repeat until the algorithm converges

The Newton’s Method: – standard technique for convex optimization problems– The Newton’s direction is found by solving

31

The Newton’s Method

It turns out is not guaranteed to be positive definite.

We are facing a challenging non-convex optimizationproblem !

is not applicable, since it is not guaranteed to be an improving direction !

We propose the Modified Inverse Iteration Algorithm.

32

Modified Inverse Iteration Algorithm

∆q is an improving direction iff

If we choose ∆Q such that

33

>0

then ∆q is an improving direction

Key Step

It turns out that this direction leads to a bound achieving solution for DOA estimation!!


After some manipulations, the condition

Note that the Newton’s method has a similar structure

34

where


where


Note that

When conditions 1. and 2. both hold, then

35

N goes to infinity1.

2.

When condition 1. and 2. both bold, the modified inverse iteration algorithm is essentially the Newton’s method!!


In practice,– N is always limited, likely to be small– The initial estimates can never be perfect

As a result, the modified inverse iteration algorithm can be considered as a novel modification of the Newton’s method for the considered non-convex optimization problem.

36

Simulation

37

Number of snapshots (N)

RM

SE

(deg

)

RM

SE

(deg

)SNR(dB)

sto-MLE (uniform)PD method sto MLE (2nd iteration) AML methodsto CRB

sto-MLE (uniform)PD method sto MLE (2nd iteration) AML methodsto CRB

Algorithms Multiple-dimensional searches ComplexityPD One M-D searchAML One M-D searchSto-MLE Two M-D search + one P-D search

Conclusions

In this talk, discussed some of the important issues in the theory and applications of sensor array signal processing– Birds localization and vocalization enhancement

● Design trade-offs (accuracy, robustness, complexity) ● Results from the real-life experiment has been demonstrated

– Stochastic ML DOA estimation in the presence of unknown non-uniform noise● Stepwise concentration & modified inverse iteration algorithm● Achieves the CRB with moderate complexity● Best performance when N is small

38

Thank you for your attention

39

theory and applications of parametric estimation methods for

Documents