single pixel imaging of laboratory and natural light...

UCSD Photonics

Photonics Systems Integration Lab

Computational Optical Sensing and Imaging 2013

Single Pixel Imaging of Laboratory and Natural Light Scenes

Stephen J. Olivas1

Yaron Rachlin2, Lydia Gu2, Brian Gardiner2, Robin Dawson2, Juha-Pekka Laine2, & Joseph Ford1

1Electrical & Computer Engineering Department University of California, San Diego

2Charles Stark Draper Laboratory, Cambridge, MA

UCSD Photonics

Samples taken all at once Exposure time

Compressive Imaging – Overview

scene lens NxN pixel detector

scene lens NxN mask single detector

Traditional Camera

Masked Sensor

Samples taken one at a time Exposure time Large single detector: Faster or specialty sensor (i.e. Photo multiplier tube, Avalanche Photodiode, infrared) Other benefits: image deblurring, depth information

8/8/2013 PHOTONIC SYSTEMS INTEGRATION LABORATORY - UCSD JACOBS SCHOOL OF ENGINEERING 2

UCSD Photonics





Traditional Camera

Masked Sensor



UCSD Photonics Compressive Imaging – Overview


Compressive Imaging

Samples taken one at a time Less samples needed if the image has a sparse representation


Jun Ke, Premchandra Shankar, and Mark A. Neifeld, “Distributed imaging using an array of compressive cameras,” Optics Communications, Vol.282, pp.185-197, 2008.

Duarte, M.F.; Davenport, M.A.; Takhar, D.; Laska, J.N.; Ting Sun; Kelly, K.F.; Baraniuk, R.G., "Single-Pixel Imaging via Compressive Sampling," Signal Processing Magazine, IEEE , vol.25, no.2, pp.83-91, March 2008

=



Compressive Imaging





=

UCSD Photonics Compressive Imaging Hardware – Background

Acquires an image by processing sampled projections Signal can be expressed as a linear combination of basis functions

Measurements Y are taken over the set of basis functions (using an DMD)

If S is sparse then only M measurements are need for perfect reconstruction

Richard G. Baraniuk – Rice University, Houston TX “Compressive Imaging,” IEEE Signal Processing pg 118 July 2007

Advantages of Compressive Sensing Low power

Encoder light / decoder heavy

Signal Acquisition and Compression are Combined

Low pixel count exotic sensors for low light or out of band λ’s

Sample Signal Rate proportional to information (Not BW)


1st Experimental Demo – Rice Univ.

Objective: Perform a systematic test of a hardware CI system

Texas Instrument Digital Mirror Device (DMD) display

UCSD Photonics

Single Pixel Camera • Transform set displayed on DMD illuminated area resolution 1,048,576 pixels (1MPix) • Image output onto detector

GOALS 1) Compare multiple transform basis sets 2) Compare functionality and performance of optics / electronics / algorithms

Single Pixel Camera System Diagram Optics Design

Backlight illuminated object (variable, incoherent, steady) Lenses (collect most light, non-obstructive, resolution) Digital Mirror Device (DMD)

DMD Specs (timing, speed, resolution, size, memory) Detector w/ DAQ

Signal acquisition (timing, speed, bit depth, sensitivity) System Integration

Custom Mounts (alignment, variable) “Solid Works”

C++ code for DMD & DAQ synchronous operation Transform Generation & Reconstruction code

Linear sum reconstruction

Total Variation reconstruction

L1 reconstruction (NESTA solver)

S. Becker, J. Bobin, and E. J. Candès, NESTA: a fast and accurate first-order method for sparse recovery.

SIAM J. on Imaging Sciences 4 (1), 1-39. 8/8/2013 PHOTONIC SYSTEMS INTEGRATION LABORATORY - UCSD JACOBS SCHOOL OF ENGINEERING 9

UCSD Photonics

Discrete/Continuous Discrete Cosine Transform (DCT) (8-bit) Wavelets, Chirplets


Binary Hadamard Noiselets

Hadamard Transform

Discrete Cosine Transform

+1

-1

0

Noiselet Transform

Transform basis set made up of 1,048,576 patterns to describe 1MPix image space

𝑺 = 𝒇−Ɣ where 1.8< Ɣ < 𝟐.𝟐

Natural Images contain more low spatial frequencies: Power Law Daniel L. Ruderman, ”Origins of Scaling in Natural Images,” Vision Research, 37 (23), 3385–3398 (1997). R. P. Millane, S. Alzaidi, and W. H. Hsiao, ”Scaling and power spectra of natural images,” in Proceedings of Image and Vision Computing New Zealand, D. G. Bailey, ed. (Massey University, Palmerston North, New Zealand, 2003), 148-153. W. H. Hsiao and R. P. Millane, ”Effects of occlusion, edges, and scaling on the power spectra of natural images,” J. Opt. Soc. Am. A, 22 (9), 1789–1797 (2005). Frenkel, G. and Katzav, E. and Schwartz, M. and Sochen, N., ”Distribution of anomalous exponents of natural images,” Phys. Rev. Lett., 97 (10), 103902 – 103906 (2006).

Compressive Imaging – Transform Basis Sets

UCSD Photonics Compressive Imaging Hardware – DMD grayscale operation

Average each subframe & scale by place value

Average all values in entire frame

Erro

r R

amp

Appr

oxim

atio

n

What is the best sampling technique to measure coefficients during Pulse Width Modulation?

The DMD uses Pulse Width Modulation to represent grayscale basis patterns


DCT

UCSD Photonics

Single-Pixel Compressive Imaging Built testbed & demonstrated working single sensor system Higher resolution & smaller physical volume than previously published work Timing / illumination artifacts need further investigation

Compressive Imaging – Recent Results

4,096 pixels (Rice 2006)

1.04MPix (UCSD-Draper)


0.7MPix (Inview Corp.) (2012)

M. F. Duarte, M. A. Davenport, D. Takhar, J. N. Laska, T. Sun, K. F. Kelly and R. G. Baraniuk “Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, 25 (2), 83-91 (2008). L. McMackin, M. A. Herman, B. Chatterjee and M. Weldon, “A high-resolution SWIR camera via compressed sensing," Proc. SPIE 8353, Infrared Technology and Applications XXXVIII, 835303-835313 (2012).

UCSD Photonics Compressive Imaging – Testbed Optics


Start with same optical configuration to verify performance

UCSD Photonics


Compressive Imaging – Test Scenes

Ground Truth Images Taken with Canon 5D Mark II SLR

Binary Image Grayscale Image Outdoor Color Image

UCSD Photonics


Compressive Imaging of Resolution Targets

Noiselet Transform

0.1% 1% 10% 100%

Hadamard Transform

DCT Transform

0.2212 0.1176 0.0480

0.1797 0.0909 0.0330

0.1912 0.1892 0.1125

UCSD Photonics


Compressive Imaging of Grayscale Scene 1% 10% 100%

Noiselet Transform

Hadamard Transform

DCT Transform

0.0774 0.0625

0.0647 0.0596

0.2383 0.1450

UCSD Photonics


Compressive Imaging of Natural Light Scene

1% Noiselet Transform

Stac

ked

RG

B Im

ages

In

frar

ed Im

ages

(650

-110

0nm

)

Filter the single pixel camera to form color images & infrared image

UCSD Photonics


Compressive Imaging of Natural Light Scene

1% Noiselet Transform 1% Hadamard Transform 1% DCT Transform

Stac

ked

RG

B Im

ages

In

frar

ed Im

ages

(650

-110

0nm

)

UCSD Photonics


Canon downsampled 1% resolution (10,000 pixels)

Canon 1% resolution

Upsampled

Compressive Imager (10,000 samples)

using 1% Hadamard Transform

Comparison to Conventional Focal Plane Imager

UCSD Photonics



Canon 1% resolution

Upsampled




UCSD Photonics

𝑆 = 𝑓−Ɣ where 1.8< Ɣ < 2.2 Power Law: Ensemble of natural images contain more low-spatial frequencies. Generally true?

Noiselet sample spatial frequency randomly Hadamard & DCT transforms can be used to target specific spatial frequencies

𝑙𝑙𝑙 𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓 ℎ𝑖𝑖ℎ𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓𝑓


Natural Images and the Power Law

UCSD Photonics



Canon 1% resolution

Upsampled


using 1% Hadamard transform


UCSD Photonics


Compressive Imaging of Point Sources 0.1% 1% 10% 100%

Noiselet Transform

Hadamard Transform

DCT Transform

UCSD Photonics



Canon 1% resolution

Upsampled



Hadamard locates bright star’s location, dim star not located


UCSD Photonics



Canon 1% resolution

Upsampled


using 1% Noiselet Transform

Noiselet locates star’s location with higher resolution, low SNR on dim star


UCSD Photonics Conclusion: Single-pixel Compressive Imagers

Acknowledgements


S. Olivas, Y. Rachlin, L. Gu, B. Gardiner, R. Dawson, J.P. Laine and J. Ford, “Single Compressive Imaging of Laboratory and Natural Light Scenes," in Computational Optical

Sensing and Imaging (2012), CTu1C.2.

S. Olivas, Y. Rachlin, L. Gu, B. Gardiner, R. Dawson, J.P. Laine and J. Ford, “Characterization of a Compressive Imaging System using Laboratory and Natural Light

Scenes," Appl. Opt. 52 19 4515-4526 (2013).

Built an experimental single pixel camera for performance testing • Solve using Linear sum, Least Squares or Total Variation

Characterize basis functions for compatibility with scene and hardware

Current configuration is not practical as a camera since it is slow

Table: 1% operation 10,486 basis patterns and measurements

Thank Charles Stark Draper Laboratory for funding under the University R&D program Robin Dawson, JP Laine, Christopher Yu, Brian Gardner, Lydia Gu, Yaron Rachlin, and Piotr Indyk for collaboration.

Compressive imaging produces images of comparable quality as traditional cameras

Compressive imaging’s main benefit lies in Feature Specific Imaging where a small number of basis functions are needed to reach a conclusive measurement.

single pixel imaging of laboratory and natural light...

Documents