compressive sensing - indico · radar imaging shallow-water acoustics video and ozone density...

Compressive Sensing

Isidora Stankovic

1 Faculty of Electrical Engineering, University of Montenegro2 GIPSA Lab, INP Grenoble, University of Grenoble Alpes

2nd Conference on Machine Learning for Gravitational Waves,Geophysics and Control Systems

September 2019, Rijeka, Croatia

www.gipsa-lab.grenoble-inp.fr

www.tfsa.ac.me



www.tfsa.ac.me

GIPSA Lab & TFSA Group

Grenoble Image Parole Signal AutomatiquePart of INP Grenoble, CNRS, University of Grenoble Alpes350 people, including 150 permanent staff and 150 PhD studentsInternationally recognized for research in automatic control, signaland image processing, speech and cognitionWeb site: www.gipsa-lab.grenoble-inp.fr

Time Frequency Signal AnalysisPart of Faculty of Electrical Engineering, University of MontenegroGroup of approx. 15 researchers with more than 300 journalpapers, together cited more than 10 000 timesTopics covering: fundamental theory of TFSA, applications such asradar and sonar signal processing, compressive sensing, etc.Web site: www.tfsa.ac.me

I. Stankovic (G2net) Compressive Sensing 11 September, 2019 2 / 35


www.tfsa.ac.me

Presentation outline

1 Introduction

2 Compressive SensingSignal reconstructionGradient reconstruction algorithmApplications

Image processingImpulse noise removalRadar imagingShallow-water acousticsVideo and Ozone density

Effects of nonsparsity

3 Conclusions


Introduction

Introduction

A sparse signal is a signal with very few non-zero components incomparison with the total length.Sparse signals can be reconstructed by the theory of compressivesensing with less measurements than with traditionalShannon-Nyquist theorem.Advantages: reduce complexity, improve efficiency (time, storage,etc), can be widely used in many fields.

0 20 40 60 80 100 120−2

0

2

Shannon−Nyquist Sampling

Time

Am

plit

ude

0 20 40 60 80 100 120−2

0

2

Compressive Sensing

Time

Am

plit

ude


Compressive Sensing Signal reconstruction

Outline

1 Introduction




3 Conclusions



Sparse signals

Consider a complex-valued discrete signal x, of length N and itstransform doman denoted by X.

We assume that X is K-sparse vector, K ¿ N. In this case, wecan use M < N samples to reconstruct the whole signal. Thevector with M available samples is denoted by y.

Ideally, measurement y is a linear combination of signalcoefficients with weighting factors ak (such as DFT matrix)

y=N−1∑k=0

ak Xk = a0X0 +a1X1 +a2X2 +·· ·+aN−1XN−1.

In compressive sensing, signal are measured M times (withdifferent weighting coefficients).



Problem definition

Unavailable samples: desired sampling technique, corruptedsamples, hardaware constraint, etc.The goal of compressive sensing is to minimize the sparsity of Xby knowing a reduced set of the available samples y. Thereconstruction can be formulated as L0-norm optimizationproblem:

min‖X‖0 subject to y=AX

where A is a measurement matrix obtained from the transformmatrix by keeping rows that corresponds to the available samples.Disadvantage: computationally inefficient, sensitive to noise.In reality, general compressive sensing formulation:

min‖X‖1 subject to y=AX



Reconstruction methods

Norm-zero based reconstructionDirect searchEstimation of the non-zero coefficients positionsMatching pursuit algorithms, OMP, CoSaMP

Norm-one based reconstructionLASSO minimizationL1-magicGradient algorithmTotal variations

Bayesian based reconstruction



Direct search procedure

In the direct search we try with all possible combinations ofnonzero index values k ∈ {k1,k2, ...,kK }=KThe vector XK contains assumed K nonzero elements of X at thepositions K. The corresponding system with M > K equations issolved in a least square sense

y=AKXK =⇒ XK =(AH

K AK

)−1AH

K y.

For all solutions we check the error y−AKXK . If the error is zerothe reconstruction is successful. If there is more than one result,then the reconstruction is not unique.

The total number of systems that should be solved is(N

K)

→ not feasible.


Compressive Sensing Gradient reconstruction algorithm

Outline

1 Introduction




3 Conclusions



Gradient algorithm – basic idea

Assume that we have reduced set of samples (measurements),and that there exist meaningful full set of samples.

We can consider missing samples as variables and try to minimizesparsity measure iteratively by using measure gradient estimation.

Gradient is estimated by using finite difference approach. Eachmissing sample is varied by ±∆ and sparsity measure is calculatedin both cases.

Gradient is proportional to the difference of sparsity measures.

The procedure is repeated for each missing sample. At the end,missing samples are updated according to the calculatedgradients.

Whole procedure is repeated. When gradient vector start tooscillate we can conclude that step ∆ should be reduced.



Algorithm

Input:Set of missing/omitted sample positions QAvailable samples (measurements) yTransformation matrix ΦStep α

Output:Reconstructed signal vector XFull set of measurements x


1: m ← 0, Set initial estimate signal vector x(0) as x(0)(ni)= y(i) for ni ∈Mand x(0)(ni)= 0 for ni ∈Q

2: ∆←maxn

|x(0)(n)|3: repeat4: repeat5: x(m+1) ← x(m)

6: for ni ∈Q do . for each missing sample7: z1 ← x(m)

8: z1(ni)← z1(ni)+∆ . try increased value9: z2 ← x(m)

10: z2(ni)← z2(ni)−∆ . try decreased value11: g(ni)←‖Φz1‖1 −‖Φz2‖1 . measure difference

12: x(m+1)(ni)← x(m)(ni)−α g(ni)13: end for14: m ← m+115: until stopping criterion is satisfied . oscillations16: ∆←∆/3 . reduce ∆17: until required precision is achieved18: x← x(m) , X←Φx


Gradient algorithm – two missing samples

−3

−2

−1

0

1

−3−2

−10

1

2

2.5

3

3.5

4

4.5

5

y(n1)

y(n2)

3.5

3.5

4

4

4

4.5

4.5

4.5

5

55.5

5.5

−4 −3 −2 −1 0 1

−1

−0.5

0

0.5

1

1.5

2

2.5

3

3.5

4



Gradient algorithm – two missing samples

Signal sparse in the DFT domain isconsidered.

The reconstruction error decay rapidlyuntil minimum for a given step ∆ isreached. That should be detectedand we should reduce finite differencestep ∆.

It can be shown that reconstructionerror (precision) is proportional to ∆.

We could use small ∆ from thebeginning, but the number ofiterations will be huge.

0 2 4 6 8

−3

0

3

Available samples

0 20 40 60−80

−60

−40

−20

0

iteration

Reconstruted MSE in [dB]

0 2 4 6 8

−3

0

3

Reconstructed signal in 60 iteratons

time n



Gradient algorithm – example

Consider a signal with N = 64 samples, sparse in the DCT domain withsparsity 3. Assume that only M = 16 samples are available.



Gradient algorithm – example

Reconstructed signal after 100 iterations



Gradient algorithm – reconstruction error

Mean squared error of the reconstruction is presented for each iteration.Iterations when ∆ is reduced are marked with red dots.

0 10 20 30 40 50 60 70 80 90 100

-120

-100

-80

-60

-40

-20

0


Compressive Sensing Applications

Outline

1 Introduction




3 Conclusions



Image reconstruction

Most of real-world images are sparse in the 2D DCT domain.If some pixels are damaged (missing, corrupted by impulsivenoise, unavailable, bad-pixels in camera CCD sensor,. . . ) we canapply reconstruction algorithm in order to find correct intensities.Consider image with 50% available pixels. Missing (or corrupted)pixels are shown in white color.



Impulsive noise removal

When the positions of the corrupted pixels in the image are known,the reconstruction is straightforward (salt & pepper noise).Problem: is it possible to detect corrupted pixels when the noise iswithin pixel intensity range ?

Initial iteration: consider all pixels as possibly corrupted.Vary value of each pixel intensity and check the sparsity measure(in the transformation domain) of the obtained images.Pixels with highest changes in sparsity measure are probablycorrupted pixels.Select one or few of them and mark them as corrupted.Reconstruct their values (assuming that other pixels are correct)and continue search in the same manner.



Image denoising – results

50% pixels are corrupted with combined noise(salt & pepper and Gaussian)

Noisy image Reconstructed image



ISAR imaging – classical full data approach

signal after coherent processing

received signal

transmitted signal – series of M pulses

2D signal

ISAR image

2D FT

coherent integration time

pulse width



ISAR imaging – reduced number of pulses

signal after coherent processing

received signal

transmitted signal

2D signal

ISAR image

2D FT

?

coherent integration time

pulse width

× × ×

× × ×

× × ×

×

×

×



ISAR imaging – MIG example

50% pulses are omitted.The reconstruction is performed by gradient algorithm.

cross−range

range

100% available pulses

cross−range

range


cross−range

range

30% − reconstructed image

cross−range

range


cross−range

range


cross−range

range


cross−range

range




Shallow-water acoustics

Transmitted - linearly frequency modulated signal, u(n)= e jπαn2.

Propagates through anunderwater dispersivechannel

The transfer function of thereceived signal:

H( f )≈+∞∑m=1

A(m, f )pr

exp{ jkr(m, f )r}.1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2

100

200

300

400

500

600

Our goal - decompose the mode functions, which will make theproblem of detecting the transmitted signal straightforward.



Video and Ozone density

Figure: captionI. Stankovic (G2net) Compressive Sensing 11 September, 2019 28 / 35

Compressive Sensing Effects of nonsparsity

Outline

1 Introduction




3 Conclusions



Influence of missing samples

The initial STFT is calculated using the available samples only

SN0(n,k)=M∑

i=1x(n+mi)w(mi)e− j 2π

N mik

0 20 40 60 80 100 120

-2

0

2

0 20 40 60 80 100 120

0

20

40

60

80

0 20 40 60 80 100 120

-2

0

2

0 20 40 60 80 100 120

0

20

40

60

80

0 20 40 60 80 100 120

-2

0

2

0 20 40 60 80 100 120

0

20

40

60

80



Nonsparse signal reconstruction

Energy in the reconstructed signal is proportional to the energy ofnonreconstructed components (sum of squared amplitudes)

‖SNR(n)−SNK (n)‖22 = K

NM

MN‖SN(n)−SNK (n)‖2

2 .

50 100 150 200 250

0

50

100

150

200

250

300

350



Example

Two linear frequency modulated signal components

Hamming window N = 256, M = 192 available samples

Reconstructed using orthogonal matching pursuit algorithm

0 100 2000

16

32

0

0.2

0.4

time

frequency 0 100 2000

16

32

0

0.2

0.4

time

frequency

0 100 2000

16

32

0

0.2

0.4

time

frequency 0 100 2000

16

32

0

0.2

0.4

time

frequency

Figure: Original (top left); K=8 (top right); K=16 (bottom left); K=32(bottom right)



Example - train

Assumed sparsity K = 55 and 50% available samples

Original STFT

10 20 30 40 50

100

200

300

400

500

Available STFT samples

10 20 30 40 50

100

200

300

400

500

Reconstructed STFT

10 20 30 40 50

100

200

300

400

5000 10 20 30 40 50 60

−20

−15

−10

−5

0

5Total error in signal

To

tal

erro

r in

dB

Assumed sparsity K


Conclusions

Summary and conclusions

Acquisition technique = relatable to many fields.Robust algorithms for successful reconstruction of unavailablemeasurements.Possible tool for signal/image denoising.Future in gravitational waves?

R. Inta, “Sparse Methods for Gravitational Wave Detection,” The Australian National University


Conclusions

Compressive Sensing

Thank you for your attention.

Questions?


compressive sensing - indico · radar imaging shallow-water acoustics video and ozone density...

Documents