compressed sensing… m.e. davieshomepages.inf.ed.ac.uk › ckiw › rpml › sparserepandcs.pdf ·...

Page 1 of 14

Compressed Sensing… M.E. Davies

Sparse representations and compressed sensing

Mike DaviesInstitute for Digital Communications (IDCOM) &

Joint Research Institute for Signal and Image Processing University of Edinburgh

with thanks to

Thomas Blumensath, Gabriel Rilling

Page 2 of 14

Compressed Sensing… M.E. Davies

Sparsity: formal definition

A vector x is K-sparse, if only K of its elements are non-zero.

In the real world “exact” sparseness is uncommon, however, many signals are “approximately” K-sparse. That is, there is a K-sparse vector xK, such that the error k x − x Kk2 is small.

Page 3 of 14

Compressed Sensing… M.E. Davies

Why Sparsity?Why does sparsity make for a good transform?

50 100 150 200 250

50

100

150

200

250

“TOM” image Wavelet Domain

Good representations are efficient - Sparse!

Page 4 of 14

Compressed Sensing… M.E. Davies

Efficient transform domain representations imply that our signals of interest live in a very small set.

Signals of interest

Sparse signal modelL2 ball (not sparse)

Page 5 of 14

Compressed Sensing… M.E. Davies

Sparse signal models can be used to help solve various ill-posed linear inverse problems:

Sparsity and ill-posed inverse problems

Missing Data RecoveryImage De-blurring

Page 6 of 14

Compressed Sensing… M.E. Davies

Sparse Representations and Generalized Sampling:compressed sensing

Page 7 of 14

Compressed Sensing… M.E. Davies

Compressed sensingTraditionally when compressing a signal we take lots of samples move to a transform domain and then throw most of the coefficients away!

Why can’t we just sample signals at the “Information Rate”?

E. Candès, J. Romberg, and T. Tao, “Robust Uncertainty principles: Exact signal reconstruction from highly incomplete frequency information,”IEEE Trans. Information Theory, 2006

D. Donoho, “Compressed sensing,” IEEE Trans. Information Theory, 2006

This is the philosophy of Compressed Sensing…

Page 8 of 14

Compressed Sensing… M.E. Davies

Signal space ~ RN

Set of signals of interest

Observation space ~ RM

M<<N

Linear projection (observation)

Nonlinear Approximation (reconstruction)

Compressed sensing

Compressed Sensing uses nonlinear reconstruction to invert the linear projection operator

Page 9 of 14

Compressed Sensing… M.E. Davies

Compressed sensingCompressed Sensing Challenges:

• Question 1: In which domain is the signal sparse (if at all)?Many natural signals are sparse in some time-frequency or space scale representation

• Question 2: How should we take good measurements?Current theory suggests that a random element in the sampling process is important

• Question 3: How many measurements do we need?Compressed sensing theory provides strong bounds on this as a function of the sparsity

• Question 4: How can we reconstruct the original signal from the measurements?

Compressed sensing concentrates of algorithmic solutions with provable performance and provably good complexity

Page 10 of 14

Compressed Sensing… M.E. Davies

Compressed sensing principle

sparse “Tom”

4

Invert transform

X =

2

Observed data

roughly equivalent

Wavelet image

Nonlinear Reconstruction

3

50 100 150 200 250

50

100

150

200

250

original “Tom”

Sparsifyingtransform

Wavelet image

50 100 150 200 250

50

100

150

200

250

1

Note that 1 is a redundant representation of 2

Page 11 of 14

Compressed Sensing… M.E. Davies

MRI acquisition

Logan-Shepp phantom We sample in this domain

Spatial Fourier Transform

Haar Wavelet Transform

Logan-Shepp phantom Sparse in this domain

Haar Wavelet Transform

Logan-Shepp phantom

Sub-sampled Fourier Transform

≈ 7 x down sampled (no longer invertible)

…but we wish to sample here

Compressed Sensing ideas can be applied to reduced sampling in Magnetic Resonance Imaging:

• MRI samples lines of spatial frequency

• Each line takes time & heats up the patient!

The Logan-Shepp phantom image illustrates this:

Page 12 of 14

Compressed Sensing… M.E. Davies

However what we really want to tackle problems like this…

originalLinear reconstruction (5x under-sampled)

Nonlinear reconstruction (5x under-sampled)

Rapid dynamic MRI acquisition in practice

(data courtesy of Ian Marshall & Terry Tao, SFC Brain Imaging Centre)

Page 13 of 14

Compressed Sensing… M.E. Davies

Synthetic Aperture RadarCompressed Sensing ideas can also be applied to reduced sampling in Synthetic Aperture Radar: Samples lines from spatial Fourier transform. Sub-sampling lines allows adaptive antennas to multi-task (interrupted SAR)

Page 14 of 14

Compressed Sensing… M.E. Davies

Potential applications…

Compressed Sensing provides a new way of thinking about signal acquisition. Potential applications areas include:

• Medical imaging

• Distributed sensing

• Seismic imaging

• Remote sensing (e.g. Synthetic Aperture Radar)

• Acoustic array processing (source separation)

• High rate analogue-to-digital conversion (DARPA A2I research program)

• The single pixel camera (novel Terahertz imaging)