computable performance analysis of sparsity recovery with...

CSSIP-INSPIRE

Computable Performance Analysis of SparsityRecovery with Applications

Arye Nehorai

Preston M. Green Department of Electrical & Systems Engineering

Washington University in St. Louis, USA

European Signal Processing Conference (EUSIPCO)

August 28, 2012

Computable Bounds 1

CSSIP-INSPIRE

Acknowledgements

• Based on collaborations with Gongguo Tang (Ph.D. 2011) and Satyabrata Sen(Ph.D 2010).

• Supported by the US National Science Foundation, Air Force Office of ScientificResearch, and the Office of Naval Research.

Computable Bounds 2

CSSIP-INSPIRE

Figure Acknowledgements

• Figures on slide 8 and slide 16 are adapted from R. Baraniuk, J. Romberg andM. Wakin’s slides “Tutorial on Compressive Sensing.”

• Figures on slide 9 and slide 10 are modified from E. J. Candes, J. Rombergand T. Tao, “Robust uncertainty principles: exact signal reconstruction fromhighly incomplete frequency information,” IEEE Trans. Inf. Theory, vol. 52,no. 2, pp. 489-509, Feb. 2006.

• Figures on slide 11 and slide 12 are reproduced from J. Wright, A. Yang,A. Ganesh, S. Sastry, and Y. Ma, “Robust face recognition via sparserepresentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no.2, Feb. 2009.

• Figure on slide 14 is from D. M. Maliouto, “A sparse signal reconstructionperspective for source localization with sensor arrays,” Master thesis, MIT,2003.

Computable Bounds 3

CSSIP-INSPIRE

Outline

• Introduction

• Sparsity recovery

• Application examples

• Future work

Computable Bounds 4

CSSIP-INSPIRE

Outline

• Introduction



• Future work

Computable Bounds 5

CSSIP-INSPIRE

Introduction

Low-dimensional structures are ubiquitous in signals:

• Sparse vectors– Compressive sensing– MRI– Image processing and computer vision

• Block-sparse vectors– Radar– Sensor array processing

• Low-rank matrices– Collaborative filtering– Robust principal component analysis

• Low-dimensional manifolds– Subspace learning– Manifold learning

Exploiting low-dimensional structures enables more accurate signal recovery.

Computable Bounds 6

CSSIP-INSPIRE

Sparsity Example: Compressive Sensing

• Interest in exploiting sparsity grew recently due to developments in compressivesensing.

• Traditional signal sampling acquires a signal using expensive hi-fidelity sensors,then compresses the data with a loss of fidelity.

• Compressive sensing (CS) combines the acquisition with the compression bysampling the signal in a novel way with less data.

• CS replaces samples with general linear projections, and linear reconstructionwith non-linear reconstruction, thus shifting the burden from the hi-fidelitysensing to reconstruction.

• Key assumption: Many natural signals x have sparse representations in sometransform domains Φ, i.e., x = Φs for some sparse vector s.

Computable Bounds 7

CSSIP-INSPIRE

Sparsity Example: Compressive Sensing (cont.)

Figure 1: Paradigm of compressive sensing.

• Surprising fact1,2: Suffices to use m = O(k log n) � n linear, non-adaptive,random measurements y to reconstruct a sparse signal, where k = ‖s‖0 is thesparsity level of s.

• The reconstruction performance depends on the sensing matrix A.

1E. J. Candes, J. Romberg and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highlyincomplete frequency information,” IEEE IT, vol. 52, no. 2, pp. 489-509, Feb. 2006.

2D. L. Donoho, “Compressed sensing,” IEEE IT, vol. 52, no. 4, pp. 1289–1306, Apr. 2006.

Computable Bounds 8

CSSIP-INSPIRE

Sparsity Example: MRI

• MRI images can often be well approximated by piece-wise constant functions.

• Instead of observing the image directly, we observe its Fourier transformcoefficients sampled along, e.g., a radial trajectory in the 2D spatial frequencydomain.

(a) Logan-Shepp phantom (b) Fourier transform (c) Sampling trajectory

Computable Bounds 9

CSSIP-INSPIRE

Sparsity Example: MRI (cont.)

• Reconstruction using minimum energy (or `2 norm) results in many artifacts.

• However, the total variation (TV) minimization, which enforces the piece-wiseconstant property of the image, recovers the original image exactly.

(d) Min-energy recovery (e) Magnitude of gradient (f) Min-TV recovery

Figure 2: Exploiting sparsity improves the MRI recovery.

Computable Bounds 10

CSSIP-INSPIRE

Sparsity Example: Image Processing and Computer Vision

• Sparse recovery is also useful in single-image super-resolution, and facerecognition3.

• In face recognition, a given facial image is sparsely represented using adictionary database.

• The significant coefficients in the representation reveals the person’s identity.

Figure 3: Robust face recognition with occlusion.

3J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust Face Recognition via Sparse Representation,”IEEE PAMI, vol.31, no.2, pp.210-227, Feb. 2009


CSSIP-INSPIRE

Sparsity Example: Image Processing and Computer Vision(cont.)

• This approach yields accurate recognition.

• It is also robust to occlusions and noise corruptions.

Figure 4: Robust face recognition with corruption.


CSSIP-INSPIRE

Sparsity Example: Sensor Arrays

• We can also use sparsity to estimate continuous parameters in nonlinearmodels4.

• Consider, for example, the estimation of directions-of-arrival (DOAs) using asensor array.

• The narrowband observation model is given by y = A(θ)x+w, where A(θ)is the array manifold matrix, and x is the signal vector.

• The unknown DOA parameter θ is continuous.

4D. Malioutov, M. Cetin, and A. S. Willsky, “A sparse signal reconstruction perspective for source localizationwith sensor arrays,” IEEE TSP, vol. 53, no. 8, pp. 3010–3022, 2005.


CSSIP-INSPIRE

Sparsity Example: Sensor Arrays (cont.)

• We discretize the parameter space into grid points given as θ = [θ1, . . . , θN ]T

to create a sparse recovery problem.

(a) DOAs of two sources. (b) DOA discretization.

Figure 5: Sparse modeling for DOA estimation.

• The observation model then becomes linear:

y = [A(θ1) · · · A(θN)]x + w = Ax + w,

where the entries of x are nonzero if and only if there is a source direction atthe corresponding grid point, implying x is sparse.


CSSIP-INSPIRE

Sparsity Example: Sensor Arrays (cont.)

Figure 6: Super resolution results.

• Discretization leads to super resolution when there is no basis mismatch.

• Sensing matrix depends on the sensor configuration and discretization strategy.


CSSIP-INSPIRE

Common Theme

• The sparse signal x is observed by alinear model corrupted by noise:

y = Ax+w.

• The sensing matrix A ∈ Rm×nprojects the set of sparse vectors inRn onto a low dimensional space Rm.

• There exist convex programs thatexploit sparsity to recover the signal.

• The recovery performance highlydepends on the sensing matrix A.

• A good matrix A should preserve thestructure of the set of sparse vectors.

Our goal: Find computable bounds on recovery errors for a given A.


CSSIP-INSPIRE

Motivations for Computable Performance Analysis

A computable performance analysis would enable us to:

• Quantify the confidence in the reconstructed signal, especially when there areno other ways to justify the correctness of the reconstructed signal.

• Optimize the system design.

Figure 6: Two MRI sampling trajectories. Left: radial, Right: spiral.


CSSIP-INSPIRE

Our Contributions

• Introduce a family of functions that quantify the goodness of sensing matricesin sparsity recovery.

• Derive bounds on reconstruction error in terms of these goodness measures forrecovery algorithms.

• Design efficient algorithms to compute these goodness measures and bounds.


CSSIP-INSPIRE

Outline

• Introduction



• Future work


CSSIP-INSPIRE

Model

Consider the measurement model

y = Ax+w,

where

• the signal x ∈ Rn is sparse with`0-sparsity level ‖x‖0 = k � n,

• the matrix A ∈ Rm×n has m rowsand n columns with m� n,

• the noise vector w ∈ Rm is either– bounded ‖w‖� ≤ ε, where� = 1, 2,∞ or

– Gaussian w ∼ N (0, σ2I).


CSSIP-INSPIRE

Sparse Signal Recovery

• In the absence of noise, the signal x can be recovered by solving

P0 : arg minx‖x‖0 subject to y = Ax.

• P0 is a non-convex optimization problem and it is NP hard to solve5.

• Convex relaxation methods replace the `0 norm with the `1 norm.

5B. K. Natarajan, “Sparse approximate solutions to linear systems,” SIAM J. on Computing, vol. 24, no. 2, pp.227− 234, 1995.


CSSIP-INSPIRE

Existing Recovery Algorithms

• Basis Pursuit: minz∈Rn ‖z‖1 subject to ‖y −Az‖� ≤ ε

• Dantzig Selector: minz∈Rn ‖z‖1 subject to ‖AT (y −Az)‖∞ ≤ λ

• LASSO Estimator: minz∈Rn12‖y −Az‖

22 + λ‖z‖1

Figure 6: Geometry of `1 minimization in the noise-free case.


CSSIP-INSPIRE

Previous Approaches for Performance Analysis

Restricted Isometry Constant (RIC)6:

δk(A) = maxz:z 6=0

∣∣‖Az‖22/‖z‖22 − 1∣∣ subject to ‖z‖0 ≤ k,

– error bounds: If ‖w‖2 ≤ ε, then the recovery error of the Basis Pursuit isbounded as

‖x− x‖2 ≤4√

1 + δ2k(A)

1− (1 +√

2)δ2k(A)ε,

– computational difficulty: No practical way to compute δk(A) exactly.

6E. J. Candes, T. Tao, “Near-optimal signal recovery from random projections and universal encoding strategies”,IEEE Trans. Inform. Theory, vol. 52, no. 12, pp. 5406-5425, Dec. 2006.


CSSIP-INSPIRE

Previous Approaches for Performance Analysis (cont.)

Mutual Coherence (MC)7:

µ(A) = maxi6=j

|ATi Aj|‖Ai‖2‖Aj‖2

,

– sufficient condition: if ‖x‖0 ≤ 12

(1 + 1

µ(A)

), then we get exact recovery

(noise-free case) via Basis Pursuit,

– however, this condition is weak,

– If ‖w‖2 ≤ ε, then the recovery error of the Basis Pursuit is bounded as8

‖x− x‖2 ≤2

1− µ(A)(4k − 1)ε.

7D. L. Donoho and M. Elad, “Optimally sparse representation in general (nonorthogonal) dictionaries via `1minimization”, in Proc. Nat. Aca. Sci., Vol. 100, pp. 2197-2202, Mar. 2003.

8D.L. Donoho, M. Elad, and V. Temlyakov, “Stable recovery of sparse overcomplete representations in thepresence of noise”, IEEE Trans. On Information Theory, Vol. 52, pp. 6-18, Jan. 2006.


CSSIP-INSPIRE

Related Work on Computable Performance Analysis

The following papers included sufficient and necessary conditions for exactrecovery of sparse vector in the noise-free case:

9A. Juditsky, A. Nemirovski, “On verifiable sufficient conditions for sparse signal recovery via `1

minimization,” Mathematical Programming Ser. B, vol. 127, pp. 57-88, 2011.

10A. Juditsky, F. Kilinc Karzan, A. Nemirovski, “Verifiable conditions of `1-recovery of sparse

signals with sign restrictions,” Mathematical Programming Ser. B, vol. 127, pp. 89-122, 2011.

11A. d’Aspremont, L. El Ghaoui, “Testing the nullspace property using semidefinite programming,”

Mathematical Programming Ser. B, vol. 127, pp. 123-144, 2011.

Our Innovations

• Verification: More efficient algorithms to verify exact recovery without noise.

• Computation: Computable performance bounds on recovery errors in noise.


CSSIP-INSPIRE

Quality Measure ω�(Q, s)

• For s ∈ [1, n] and A ∈ Rm×n, we define

ω�(Q, s) = minz:z 6=0

‖Qz‖�‖z‖∞

subject to‖z‖1‖z‖∞

≤ s,

where Q = A or ATA, and � = 1, 2, or ∞.

• s is a measure of the sparsity of z: Smaller s implies more sparse z; also largerω�(Q, s) and better reconstruction performance.

• Without the sparsity constraint, with`∞ norm replaced by `2 and � replacedwith `2, ω�(Q, s) is the minimalsingular value of Q.

• ω�(Q, s) is a measure of theincoherence (quality) of A, and it willdetermine the performance bounds.

Figure 7: Constraint set in R3 for s = 1.4.


CSSIP-INSPIRE

Reconstruction Error Bounds

Theorem 1. Suppose the noise w satisfies ‖w‖� ≤ ε, ‖ATw‖∞ ≤ λ, and‖ATw‖∞ ≤ κλ, κ ∈ (0, 1), for the Basis Pursuit, the Dantzig Selector, and theLASSO estimator, respectively, then we have

‖x− x‖∞ ≤2ε

ω�(A, 2k)for the Basis Pursuit,

‖x− x‖∞ ≤2λ

ω∞(ATA, 2k)for the Dantzig Selector, and

‖x− x‖∞ ≤(1 + κ)λ

ω∞(ATA, 2k/(1− κ))for the LASSO estimator.

• These error bounds are inversely proportional to ω�(Q, s).

• ω�(Q, s) > 0 implies exact recovery in the noise-free case (where ε = 0, λ = 0).

• When the sparsity level k of the signal decreases, ω�(Q, s) becomes larger,implying smaller reconstruction error.


CSSIP-INSPIRE

Reconstruction Error Bounds (cont.)

The error bounds on the `1 and `2 norms can be expressed via

‖x− x‖1 ≤ ck‖x− x‖∞ and ‖x− x‖2 ≤√ck‖x− x‖∞.


CSSIP-INSPIRE

Topics of Next Slides


CSSIP-INSPIRE

Verification of ω�(Q, s) > 0: General Case

We provide a computable way to verify sufficient conditions for exact sparserecovery in the noise-free case (see also Shtok et. al.12).

Theorem 2. Define s∗ = max{s : ω�(Q, s) > 0}. Then,

k ≤ bs∗/2c =⇒ exact sparse recovery.

In addition, s∗ is the inverse of the maximum of the n optimal values of thefollowing linear programs:

maxz

zi subject to Qz = 0, ‖z‖1 ≤ 1, i = 1, . . . , n.

Thus, bs∗/2c is the maximal sparsity level below which exact recovery isguaranteed in the noise-free case.

12J. Shtok and M. Elad, “Analysis of the Basis Pursuit Via the Capacity Sets”, Jour. of Fourier Analysis andApplications, vol. 14, no. 5-6, pp. 688-711, Dec. 2008.


CSSIP-INSPIRE

Verification of ω�(Q, s) > 0: Fourier Case

For the special yet important class of Fourier sensing matrices, the computationalcost can be greatly reduced.

Theorem 3. If H is the Fourier transform matrix on a finite abelian group,and the rows of A are sampled from the rows of H, then the optimal values of

maxz

zi subject to Qz = 0, ‖z‖1 ≤ 1

are equal for i = 1, 2, . . . , n.

• For these sensing matrices, we compute s∗ by solving a single linear program.

• Examples include the Fourier matrix and the Hadamard matrix, which arewidely used in compressive sensing


CSSIP-INSPIRE

Numerical Examples: Maximal Sparsity Levels

Table 1: Comparison of our algorithm for sufficient conditions for exact recoverywith Juditsky and Nemirovski’s in bounding the maximal sparsity levels bs∗/2cfor Gaussian sensing matrices (n = 256).

Max Sparsity Level

mbs∗/2c CPU time

Our Algorithm JN’s Algorithm Our Algorithm JN’s Algorithm

25 1 1 6 9151 2 2 8 19176 3 3 10 856

102 4 4 13 5630128 4 5 16 5711153 6 6 20 1381179 7 7 24 3356204 10 10 25 10039230 13 14 28 8332

Observation: The two algorithms give similar maximal sparsity levels, but ours ismuch faster.


CSSIP-INSPIRE

Numerical Examples: Maximal Sparsity Levels (cont.)

Table 2: Comparison of sufficient conditions for exact recovery based on ω andthe Mutual Coherence for Hadamard matrices.

Max Sparsity Level

mn

n = 2048 n = 4096 n = 8192

bs∗

2 c b12(1 + 1µ)c bs

∗

2 c b12(1 + 1µ)c bs

∗

2 c b12(1 + 1µ)c

.2 6 3 9 4 12 6

.3 9 4 13 5 18 7

.4 12 5 17 7 24 10

.5 15 6 21 9 30 13

.6 19 8 27 9 39 16

.7 25 9 35 14 50 17

.8 34 13 48 17 68 23

.9 53 20 74 22 105 34

Observation: Our sufficient condition for exact recovery is stronger than given bythe Mutual Coherence for Hadamard matrices.


CSSIP-INSPIRE

Computation of ω�(Q, s): General Case

We provide a way to compute ω�(Q, s) for any given Q and s, which readilytranslates to upper bounds on recovery errors, in the noisy case.

Theorem 4. The quantity ω�(Q, s) is the minimum of the optimal values ofthe following n linear programs or quadratic programs:

minu∈Rn−1

‖qi −Q(:,−i)u‖� s.t. ‖u‖1 ≤ s− 1, i = 1, . . . , n.

• Here qi is the ith column of Q and Q(:,−i) are columns except the ith one.

• The ith optimization finds the best approximation of the ith column using asparse (measured by `1 norm) linear combination of the rest columns.

• Thus, if the columns of Q can well approximate each other using sparse linearcombinations, ω� is small and the reconstruction error is large.

• Note, the Mutual Coherence considers the approximability between twocolumns. Our ω� is more accurate because it considers all columns.


CSSIP-INSPIRE

Computation of ω�(Q, s): Fourier Case

The computation cost can be greatly reduced for the special yet important classof Fourier sensing matrices, similar to the verification case.

Theorem 5. If H is the Fourier transform matrix on a finite abelian group,and the rows of A are sampled from the rows of H, then the optimal values of

minu∈Rn−1

‖qi −Q(:,−i)u‖� subject to ‖u‖1 ≤ s− 1

are equal for all i = 1, . . . , n.

• For these sensing matrices, we compute a single ω�(Q, s) by solving a singlelinear program or quadratic program.

• Examples include the Fourier matrix and the Hadamard matrix.


CSSIP-INSPIRE

Numerical Examples: Performance Bounds Comparison I

Table 3: ω2(A, s) based bounds ‖x− x‖2 ≤ 2√

2kω2(A,2k)ε vs. RIC based bounds

‖x− x‖2 ≤4√

1+δ2k

1−(1+√

2)δ2kε for the Basis Pursuit with Bernoulli sensing matrices

and n = 256 with ε = 1.

Bound on Estimation Errork m 51 77 102 128 154 179 205

1ω bound 4.2 3.8 3.5 3.4 3.3 3.2 3.2RIC bound 23.7 16.1 13.2 10.6 11.9

2ω bound 31.4 12.2 9.0 7.4 6.5 6.0 5.6RIC bound 72.1 192.2

3 ω bound 252.0 30.9 16.8 12.0 10.1 8.94 ω bound 52.3 23.4 16.5 13.65 ω bound 57.0 28.6 20.16 ω bound 1256.6 53.6 30.87 ω bound 161.6 50.68 ω bound 93.19 ω bound 258.7


CSSIP-INSPIRE

Numerical Examples: Performance Bounds Comparison II

Figure 8: ‖x− x‖2 ≤ 2√

2kω2(A,2k)ε vs. MC based bound ‖x− x‖2 ≤ 2

1−µ(A)(4k−1)ε

for the Basis Pursuit for Hadamard sensing matrices with n = 2048.


CSSIP-INSPIRE

Numerical Examples: Observations

• The bounds using ω� are tighter than the bounds based on Mutual Coherenceor RIC.

• Bounds based on ω� still apply even when the bounds based on MutualCoherence or RIC do not apply, e.g., for small m and large k.


CSSIP-INSPIRE

Quality Measure: l1-CMSV

• For s ∈ [1, n] and A ∈ Rm×n, we define the l1-constrained minimal singularvalue (l1-CMSV) of A by

ρs(A) = minz:z 6=0

‖Ax‖2‖x‖2

, subject to‖z‖21‖z‖22

≤ s.

• ρs(A) is approximated by solving a constrained optimization problem using aninterior point method.

• Similar to ω�(Q, s), ρs(A) is a measure of incoherence of A, and s is ameasure of the sparsity of z.

• Bounds using ρs(A) are computationally more amenable than using RIC.

• Bounds based on ρs(A) are also tighter than bounds using ω�(Q, s).


CSSIP-INSPIRE

Reconstruction Error Bounds Using l1-CMSV

Theorem 6. Suppose the noise w satisfies ‖w‖2 ≤ ε, ‖ATw‖∞ ≤ λ, and‖ATw‖∞ ≤ κλ, κ ∈ (0, 1), for the Basis Pursuit, the Dantzig Selector, andthe LASSO estimator, respectively, then we have

‖x− x‖2 ≤2ε

ρ4kfor the Basis Pursuit,

‖x− x‖2 ≤4√k

ρ24k

λ for the Dantzig Selector, and

‖x− x‖2 ≤(1 + κ)

1− κ.

2√k

ρ24k/(1−κ)2

λ for the LASSO estimator.


CSSIP-INSPIRE

Outline

• Introduction


• Application example

– Multi-objective optimization of OFDM radar waveform for target detection

• Future work


CSSIP-INSPIRE

Problem Description

• Goal: Detect a far-field target in the presence of multipath reflections.

• Challenges:

– Complex physical phenomena: multiple reflections, fading effects, etc.

– Lack of line-of-sight (LOS) propagation path to the target.

– Unknown frequency response of the target.

Our Approach

• Employ OFDM signal to increase the frequency diversity and overcome fading.

• Exploit multipath reflections to improve the spatial diversity.


CSSIP-INSPIRE

Problem Description

Radar

Targets

Buildings

Non-LOS region

Figure 9: Urban multipath scenario.


CSSIP-INSPIRE

Our Approach (cont.)

• Reformulate the target detection problem as estimating the spectrum of asparse signal by exploiting the sparsity of multiple signal paths and targetvelocity.

• Employ a sparse-recovery algorithmbased on the Dantzig selector (DS)approach and analyze its performancein terms of the l1-constrained minimalsingular value of the measurementmatrix.

• Propose a constrained multi objectiveoptimization (MOO) technique todesign the spectral parameters of theOFDM waveform. Radar

Target A

Reflecting

surface

Reflecting

surface

Target B

Target C

Image of

target A

Image of

target C

Constant

range curve

v v

v

A

B

C


CSSIP-INSPIRE

Measurement Model

Assumptions:

– Far-field, point target moving with a constant velocity.

– The target remains within a range cell over the coherent processing interval(CPI).

– Fixed, mono-static, and coherent radar.

– The radar knows the geometry of the environment and position of the rangecell.

– The information of the known range cell (τ) is incorporated into the modelby choosing t = τ + nT

PRI, n = 0, 1, . . . , N − 1.

n = 0 n = 1 n = N-1

Coherent processing interval

T

TPRI


CSSIP-INSPIRE

Measurement Model (cont.)

• Signal model: The complex envelope of the transmitted OFDM signal is

s(t) =

L−1∑l=0

al ej2πl∆ft,

where

– L : number of subcarriers,– a = [a0, a1, . . . , aL−1]

T : complex transmitted weights ensuringaHa = 1 for constant energy transmission,

– ∆f = B/(L+ 1) = 1/T : subcarrier spacing,– B : signal bandwidth,– T : pulse duration.

• Adaptive design: We will select the coefficients al’s to maximize thetarget-detection performance.


CSSIP-INSPIRE

Measurement Model (cont.)

Consider a target corresponding to the known range cell τ , and the radarreceives information about the target (moving with v) through the path p.Then, the complex envelope of the received signal at the l-th subchannel

yl(n) = al xlp φl(n, p,v) + el(n), for l = 0, . . . , L− 1, n = 0, . . . , N − 1, (1)

where

φl(n, p,v) , e−j2πflτ ej2πflβpnTPRI,

and

– xlp : target scattering coefficient at the l-th subchannel and p-th path,– βp = 2〈v,up〉/c : effective Doppler coefficient along the p-th path,– up : direction-of-arrival (DOA) unit-vector of the p-th path,– c : speed of propagation,– fl = fc + l∆f and fc is the carrier frequency,– el(n) : clutter, measurement noise, and co-channel interference (CCI).


CSSIP-INSPIRE

Sparse Model

• We discretize the possible signal paths and target velocities into P and V gridpoints, respectively.

• Considering all possible combinations of (pi,vj), i = 1, 2, . . . , P, j =1, 2, . . . , V , we can rewrite (1) as

yl(n) = alφl(n)T xl + el(n),

where

– φl(n) =[φl(n, p1,v1), . . . , φl(n, p1,vV ), φl(n, p2,v1), . . . , φl(n, pP ,vV )

]T,

– xl is a PV × 1 sparse vector, having only kl non-zero entries, where

kl = |Il| : sparsity level of xl,

Il ={i ∈ [1, P ] : pi-th path carries target information

}.


CSSIP-INSPIRE

Sparse Model (cont.)

Concatenating the measurements of all L subchannels and N time samples

y = Φx+ e,

where

– y =[y(0)T , . . . ,y(N − 1)T

]Tand y(n) = [y0(n), . . . , yL−1(n)]

T ,

– Φ =[(AΦ(0))

T · · · (AΦ(N − 1))T]T

with A = diag(a) and Φ(n) =

blkdiag(φ0(n)T , . . . ,φL−1(n)T

),

– x =[xT0 , . . . ,x

TL−1

]Tis a sparse-vector having k =

∑L−1l=0 kl non-zero

entries,

– e =[e(0)T , . . . , e(N − 1)T

]Tand e(n) = [e0(n), . . . , eL−1(n)]

T .

Thus, Φ contains the response bases corresponding to all possible paths andvelocities.


CSSIP-INSPIRE

Statistical Model

• The vector e(n) represents the clutter, measurement noise, and co-channelinterference at the output of L subchannels.

• We assume that

– e(n) is temporally white, zero-mean complex Gaussian vector,– co-channel interference among the subchannels is characterized by a

covariance matrix Σ.

• Hence, the measurement vector is distributed as

y ∼ CNLN (Φx, IN ⊗Σ) .

• The identity matrix IN is due to the temporal white noise distribution.


CSSIP-INSPIRE

Sparse Recovery and Performance

• To obtain an estimate of x (a k-sparse vector) from the noisy measurementsy, obtained through a linear model

y = Φx+ e,

we apply the Dantzig selector (DS).

• Recall that the DS is given by

xDS

= minz∈CLPV

‖z‖1 subject to∥∥ΦH (y −Φz)

∥∥∞ ≤ λ · σ, (2)

where λ =√

2 log(LPV ) is a control parameter and σ =√

tr(Σ)/L.

• To assess the reconstruction performance of this recovery algorithm, we usethe l1-constrained minimal singular value (l1-CMSV) of Φ.


CSSIP-INSPIRE

Sparse Recovery - Further Simplification• We observe an additional structure in the sparse measurement model, i.e.,

y =[

Φ0 · · · ΦL−1

] x0...

xL−1

+ e,

where– each pair of block-matrices is orthogonal, i.e., ΦH

l1Φl2 = 0 for l1 6= l2,

– each xl, l = 0, 1, . . . , L− 1, is sparse with sparsity level kl.

• To obtain an estimate of x,

x =[xT0 , . . . , x

TL−1

]T,

by exploiting these properties, we employ simpler L decomposed Dantzigselectors.

xDDS,l = minzl∈CPV

‖zl‖1 subject to∥∥ΦH

l (y −Φlzl)∥∥∞ ≤ λl · σ, (3)

where λl =√

2 log(PV ).


CSSIP-INSPIRE

Performance Analysis

Theorem 7. Consider the estimate x obtained by employing our decomposedDS. An upper bound exists on the `2-norm of the sparse-estimation error

‖xDDS − x‖2 ≤ 4

√√√√L−1∑l=0

λ2l kl σ

2

ρ44kl

(Φl),

whereas using the original DS we get

∥∥xDS− x

∥∥2≤ 4

λ√k σ

ρ24k(Φ)

.

Theorem 8. The L small Dantzig selectors in (3) perform better than theoriginal Dantzig selector in (2) in terms of a smaller upper bound on the `2-norm of the sparse-estimation error:

4

√√√√L−1∑l=0

λ2l kl σ

2

ρ44kl

(Φl)≤ 4

λ√k σ

ρ24k(Φ)

.


CSSIP-INSPIRE

Adaptive Waveform Design

• We can adaptively design the OFDM spectral parameters, al, to minimize theupper bound on the sparse-estimation error as

a(1) = arg mina∈CL

L−1∑l=0

λ2l kl σ

2

a4l ρ

44kl

(Φl)subject to aHa = 1,

where Φl = alΦl.

• However, the computation of ρ4kl(Φl) is difficult with the complex variables.

Therefore, we use a computable lower bound on ρ4kl(Φl), defined as

ρ8kl(Ψl) ≤ ρ4kl(Φl),

whereΨTl Ψl =

[ΨT

1 Ψ1 + ΨT2 Ψ2 0

0 ΨT1 Ψ1 + ΨT

2 Ψ2

],

Ψ1 = Re Φl, and Ψ2 = Im Φl.


CSSIP-INSPIRE

Adaptive Waveform Design (cont.)

• Hence, to minimize the upper bound on the sparse-estimation error, weformulate a single-objective optimization problem as

a(1) = arg mina∈CL

L−1∑l=0

λ2l kl σ

2

a4l ρ

48kl

(Ψl)subject to aHa = 1.

• Using the Lagrange-multiplier approach, we easily obtain the solution as

a(1)l =

√√√√ (2αl)1/3∑L−1

l=0 (2αl)1/3, for l = 0, 1, . . . , L− 1,

where αl =λ2l kl σ

2

ρ48kl

(Ψl).

• Note that the designed waveform depends solely on Φ, the measurementmatrix.


CSSIP-INSPIRE


• However, to achieve better detection performance it is also essential that thesignal parameters are adaptive to the target and noise parameters (x and Σ)

• To detect the presence of a target in the range cell under test, a standardprocedure is to construct the following decision problem{

H0 : y = eH1 : y = Φx+ e

,

and to find out whether the measurement y is distributed asCNLN (0, IN ⊗Σ) or CNLN (Φx, IN ⊗Σ).

• To optimize the detection performance, we maximize the squared Mahalanobis-distance between these two distributions

d2 = xH ΦH (IN ⊗Σ)−1

Φx.


CSSIP-INSPIRE


• Hence, in addition to minimizing the upper bound on the estimation error,we propose maximizing another single-objective function based on the squaredMahalanobis-distance (d2) as

a(2) = arg maxa∈CL

[xH ΦH (IN ⊗Σ)

−1Φx

]︸︷︷︸

, d2

, subject to aHa = 1.

• After some algebraic manipulation, we have

d2 = aH

[N−1∑n=0

(Φ(n)xxH Φ(n)H

)T �Σ−1

]a,

and therefore the solution of the optimization problem, a(2),will be the eigenvector corresponding to the largest eigenvalue of[∑N−1

n=0

(Φ(n)xxH Φ(n)H

)T �Σ−1].


CSSIP-INSPIRE

Multi-Objective Optimization

• We adaptively design the OFDM spectral parameters, al, using a constrainedmulti-objective optimization (MOO) that simultaneously optimizes twoobjective functions:

– minimize the upper bound on the sparse-estimation error,– maximize the squared Mahalanobis-distance.

Mathematically, this is represented as

aopt =

arg mina∈CL

∑L−1l=0

λ2l kl σ

2

a4lρ4

4kl(Φl)

,

arg maxa∈CL aH[∑N−1

n=0

(Φ(n)xxH Φ(n)H

)T �Σ−1]a

,

subject to aHa = 1.

• We employ the well-known nondominated sorting genetic algorithm II (NSGA-II) to solve our MOO problem, imposing a restriction on the solutions to satisfythe constraint aHa = 1.


CSSIP-INSPIRE

Numerical Examples

• Problem: Detect a moving target in the presence of multipath in 2D.

• Target and multipath parameters:

– The range cell that is at a distance of 3 kmfrom the radar.

– The target is 13.5 m east from the center line,moving with velocity v = (35/

√2) (i+ j) m/s

and remains within the range cell over a CPI.– There are two actual paths between the

target and radar: one direct and onereflected, subtending angles of 0.26◦ and 0.51◦,respectively, with respect to the radar.

Radar

Reflecting

surface

Reflecting

surface

TargetImage of

target Constant

range curve

v

3 km

13.5 m

40 m

– The scattering coefficients are varied to simulate three target responses:

∗ Target 1: x(1)d = [1, 1, 1]T , x

(1)r = [0.5, 0.5, 0.5]T .

∗ Target 2: x(2)d = [4, 1, 2]T , x

(2)r = [2, 0.5, 1]T .

∗ Target 3: x(3)d = [1, 10, 1]T , x

(3)r = [0.5, 5, 0.5]T .


CSSIP-INSPIRE

Numerical Examples (cont.)

Radar parameters:

– Carrier frequency fc = 1 GHz– Available bandwidth B = 100 MHz– Number of OFDM subcarriers L = 3– Subcarrier spacing of ∆f = B/(L+ 1) = 25 MHz– Pulse width T = 1/∆f = 40 ns– Pulse repetition interval T

P= 4 ms

– Number of coherent pulses N = 20– All the transmit OFDM weights were equal; i.e., al = 1/

√L ∀ l


CSSIP-INSPIRE


Simulation:

– We partition the signal paths and target velocities into P = 5 and V = 3uniform grid points.

– Hence, the associated signal grid paths subtend angles of{−0.5◦,−0.25◦, 0◦, 0.25◦, 0.5◦} with respect to the radar, and target gridvelocities are {25, 35, 45} m/s.

– Note, these grid points are different from the true parameters.

– Generate the noise samples from a CN (0, 1) distribution, and then scale tosatisfy the required target to clutter-plus-noise ratio (TCNR)

TCNR =xH x

N Lσ20

.

Comment: We kept the clutter-plus-noise power to be the same in eachsubcarrier by considering Σ = σ2

0 IL. Hence, TCNR will depend only on thetarget’s RCS at each frequency.


CSSIP-INSPIRE


Parameters of NSGA-II:

– Population size = 500– Number of generations = 50– Crossover probability = 0.9– Mutation probability = 0.1– The constraint aHa = 1 is relaxed by ensuring that the solutions satisfy

0.999 ≤ aHa ≤ 1.001


CSSIP-INSPIRE


Performance of the standard and Decomposed Dantzig Selectors:Target 1

0 5 10 15 200

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

Target to clutter plus noise ratio (TCNR) (in dB)

Nor

mal

ized

roo

t mea

n sq

ured

err

or

Original DSDecomposed DS

10−1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Probability of false alarm (PFA

)

Pro

babi

lity

of d

etct

ion

(PD)


0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Com

puta

tion

time

(in s

ec)


Target 2

0 5 10 15 200

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2


Nor

mal

ized

roo

t mea

n sq

uare

d er

ror


10−2

10−1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


)

Pro

babi

lity

of d

etec

tion

(PD)


0 5 10 15 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


Com

puta

tion

time

(in s

ec)


Normalized RMSE. Empirical ROC. Computation time.


CSSIP-INSPIRE


Solutions of the single-objective optimization problems:

– Minimizing the upper bound on the sparse-estimation error we obtaineda(1) = [0.54, 0.16, 0.83]T , irrespective of the target parameters.

– Maximizing the squared Mahalanobis-distance we found

∗ a(2) = [1, 0, 0]T or [0, 1, 0]T or [0, 0, 1]T for Target 1,

∗ a(2) = [1, 0, 0]T for Target 2,

∗ a(2) = [0, 1, 0]T for Target 3.

Observation: The maximization of the squared Mahalanobis-distance providedan adaptive waveform with all the signal energy concentrated over a singlesubcarrier that had the strongest target response, thus losing the frequencydiversity of the system.


CSSIP-INSPIRE


Solutions of the NSGA-II (MOO problem) at the 50th generation:

Solutions

0

0.5

1

0

0.5

1

0

0.2

0.4

0.6

0.8

1

|a2||a

1|

|a3|

0

0.5

1

0

0.5

1

0

0.2

0.4

0.6

0.8

1

|a2|

|a1|

|a3|

0

0.5

1

0

0.5

1

0

0.2

0.4

0.6

0.8

1

|a2||a

1|

|a3|

Pareto-front

0 2 4 6 8 10

x 109

50

100

150

200

250

300

350

Squared upper bound on sparse−estimation error

Squ

ared

Mah

alan

obis

dis

tanc

e

0 2 4 6 8 10

x 108

140

160

180

200

220

240

260


Squ

ared

Mah

alan

obis

dis

tanc

e

0 1 2 3 4 5

x 109

200

300

400

500

600

700

800

900

1000

1100


Squ

ared

Mah

alan

obis

dis

tanc

e

Target 1 Target 2 Target 3


CSSIP-INSPIRE


Effect of target scattering coefficent on the NSGA-II solutions:

We averaged the whole population of 500 solutions and found

– aopt,avg = [0.61, 0.39, 0.68]T for Target 1,

– aopt,avg = [0.88, 0.20, 0.36]T for Target 2,

– aopt,avg = [0.13, 0.96, 0.15]T for Target 3.

Observation: The solution of the MOO distributes the energy of the optimalwaveform across different subcarriers in proportion to the distribution of thetarget energy; i.e., it puts more signal energy into that particular subcarrier inwhich the target response is stronger.


CSSIP-INSPIRE


Performance improvement due to adaptive waveform design:

0 2 4 6 8 10 12 14 16 18 200

0.5

1

1.5

2


Nor

mal

ized

roo

t mea

n sq

uare

d er

ror

Fixed waveforml1−CMSV minimized adaptive waveform

NSGA−II optimized adaptive waveform

10−2

10−1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1


)

Pro

babi

lity

of d

etec

tion

(PD

)

Fixed waveforml1−CMSV minimized adaptive waveform

NSGA−II optimized adaptive waveform

RMSE ROC


CSSIP-INSPIRE

Outline

• Introduction


• Application example

• Future work


CSSIP-INSPIRE

Open Problems

• Develop computable tight bounds.

• Bounds for other signal structures, such as block-sparse vectors and low-rankmatrices.

• Procedures for optimization of system design.

• Computationally efficient algorithms for sensing matrices other than Fourier orHadamard.

• Minimization of modeling errors due to discrete grid mismatch.


CSSIP-INSPIRE

Possible Approaches

• Develop computable tighter bounds that control the average or typical systemperformance.

• Encode more signal structures into the model rather than using bounds, forexample using the continuous-parameter domain.


CSSIP-INSPIRE

References

• G. Tang and A. Nehorai, “Verifiable and computable performance analysis ofsparsity recovery,” submitted for publication.

• G. Tang and A. Nehorai, “Performance analysis of sparse recovery based onconstrained minimal singular values,” IEEE Trans. Signal Processing, vol. 59,no. 12, pp. 5734-5745, Dec. 2011.

• G. Tang and A. Nehorai, “Fixed point theory and semidefinite programmingfor computable performance analysis of block-sparsity recovery,” submitted forpublication.

• S. Sen, G. Tang, and A. Nehorai, “Multiobjective optimization of OFDM radarwaveform for target detection,” IEEE Trans. on Signal Processing, Vol. 59,pp. 639-652, Feb. 2011.


CSSIP-INSPIRE

Questions ?


CSSIP-INSPIRE

Thank You !


computable performance analysis of sparsity recovery with...

Documents