Introduction to Compressive Sensing

Richard Baraniuk, Compressive sensing . IEEE Signal Processing Magazine, 24(4), pp. 118-121, July 2007)

Emmanuel Candès and Michael Wakin, An introduction to compressive sampling . IEEE Signal Processing Magazine, 25(2), pp. 21 - 30, March 2008

A course on compressive sensing, http://w3.impa.br/~aschulz/CS/course.html

Outline

• Introduction to compressive sensing (CS)– First CS theory– Concepts and applications– Theory• Compression• Reconstruction

Introduction• Compressive sensing

– Compressed sensing– Compressive sampling

• First CS theory– E. Cand`es, J. Romberg, and T. Tao, “Robust uncertainty principles: Exact signal

reconstruction from highly incomplete frequency information,” IEEE Trans. Inform. Theory, vol. 52, no. 2, pp. 489–509, Feb. 2006.

Cand`es Romberg Tao

Compressive Sensing: concept and applications

Compression/Reconstruction

CS samplingX RNx1

RMxN

Measurement matrix

yRMx1 Quantizationhuman coding

Transmit

OptimizationInverse

transform (e.g., IDCT)

s X’InverseQuantization

human coding

y’

syss ' subjet to ||||min 0 sX '

: transform basis (e.g., DCT basis)

CS Reconstruction

Theory and Core Technologycompression

• K-sparse– most of the energy is at low frequencies– K non-zero wavelet (DCT) coefficients

sX

Compression

y XMeasurement matrix

Compression

y transform basis

scoefficient

X

Compression

y

transform basis

scoefficient

Reconstruction

Reconstruction: optimization

syss

osubject t ||||min 0

syss

osubject t ||||min 1

NP-hard problem

Linear programming [1][2]Orthogonal matching pursuit (OMP)

syss

osubject t ||||min 2

Minimum energy ≠ k-sparse

(1)

(2)

(3)

(4) Greedy algorithm [3]

Compressive sensing: significant parameters

1. What measurement matrix should we use?2. How many measurements? (M=?)3. K-sparse?

Measurement Matrix Incoherence

(1) Correlation between and ],1[),( )2( n

Examples

= noiselet, = Haar wavelet (,)=2= noiselet, = Daubechies D4 (,)=2.2= noiselet, = Daubechies D8 (,)=2.9– Noiselets are also maximally incoherent with spikes and

incoherent with the Fourier basis

= White noise (random Gaussian)

Restricted Isometry Property (RIP)preserving length

• RIP: For each integer k = 1, 2, …, define the isometry constant k of a matrix A as the smallest number such that

(1) A approximately preserves the Euclidean length of k-sparse signals

(2) Imply that k-sparse vectors cannot be in the nullspace of A

(3) All subsets of s columns taken from A are in fact nearly orthogonal– To design a sensing matrix , so that any subset of columns of size k be

approximately orthogonal.

)1(||||||||

)1( 2

2

2

2k

l

lk x

Ax

How many measurements ?

)log(~log),(2

nkOMnkCM

Single-Pixel CS Camera[Baraniuk and Kelly, et al.]

On the Interplay Between Routing and SignalRepresentation for Compressive Sensing in

Wireless Sensor Networks

G. Quer, R. Masiero, D. Munaretto, M. Rossi, J. Widmer and M. Zorzi

University of Padova, Italy.DoCoMo Euro-Labs, Germany

Information Theory and Applications Workshop (ITA 2009)

Network Scenario Settingx11 x12 x13 x14

x21 x22 x23 x24

… … .. ..… .. … …

X

Example of the considered multi-hop topology.

Irregular network setting [4](1) Graph wavelet(2) Diffusion wavelet

Measurement matrix Built on routing path

Routing path mm xy 111

random }1,1{1

mmm xyy 2212

mmm xyy 3323

mmm xyy 4434

…………………………………………

……………… ……………………

mlm

y ,........ ml321

nx

xxx

.3

2

1

Measurement matrix

• R1: is built according to routing protocol, – randomly selected from {+1, -1}

• R2: is built according to routing protocol – randomly selected from (0, 1]

• R3: has all coefficients in randomly selected from {+1, -1}• R4: has all coefficients in randomly selected from(0, 1]

Transform basis

• T1: DCT• T2: Haar Wavelet• T3: Horizontal difference• T4: Vertical difference + Horizontal difference

Degree of sparsity

DCT

Haar

H-diff

VH-diff

Incoherence

DCT

Haar

H-diffVH-diff

Performance Comparison

• Random sampling (RS)– each node sends its data with probability P = M/N,

the data packets are not processed at internal nodes but simply forwarded.

• RS-CS– the data values are combined

with that of any other node encountered along the path.

Routing path mm xy 111

random }1,1{1

mmm xyy 2212

mmm xyy 3323

mmm xyy 4434

Reconstruction Error

Reconstruction Errorpre-distribution for T3 and T4 [5]

Research issues when applying CS in Sensor Networks

1. How to construct measurement matrix – Incoherent with transform basis – Distributed – M=?

2. How to choose transformation basis – Sparsity– Incoherent with measurement matrix

3. Irregular sensor deployment– Graph wavelet– Diffusion wavelet

References[1] Bloomfield, P., Steiger, W., Least Absolute Deviations: Theory, Applications, and Algorithms. Progr. Probab. Statist. 6, Birkhäuser, Boston, MA, 1983.[2] Chen, S. S., Donoho, D. L., Saunders, M. A, Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20 (1999), 33–61.[3] J. Tropp and A. C. Gilbert, “Signal recovery from partial information via orthogonal matching pursuit,” Apr. 2005, Preprint.[4] J. Haupt, W.U. Bajwa, M. Rabbat, and R. Nowak, “Compressed

sensing for networked data,” IEEE Signal Processing Mag., vol. 25, no. 2, pp. 92-101, Mar. 2008.

[5] M. Rabbat, J. Haupt, A. Singh, and R. Novak, “Decentralized Compression and Predistribution via Randomized Gossiping,” in IPSN, 2006.