siggraph 2012 computational plenoptic imaging course - 3 spectral imaging
DESCRIPTION
SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral ImagingTRANSCRIPT
![Page 1: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/1.jpg)
Spectral Imaging
Ivo IhrkeSaarland University/MPI Informatik
![Page 2: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/2.jpg)
• spectrometer
The spectral data cube
![Page 3: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/3.jpg)
disadvantage:• dispersion relation is nonlinearadvantage:• light efficient
Principle of Operation - Dispersion
![Page 4: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/4.jpg)
Diffraction Grating
– At center, no diffraction
– For higher orders, diffraction is
taking place
Diffraction Order Percentage of transmitted Light
0 25%
1 20.26%
2 10.13%
3 2.25%
4 0 %
remainder 9.72%
disadvantage:• low light efficiencyadvantage:• linear relation pixel pos. <-> wavelength
![Page 5: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/5.jpg)
– Diffraction-based example ->
– Spectrometer calibration (all types)
1. mapping
pixel <–> wavelength
2. relative intensity of wavelengths
Diffraction-Based Systems
![Page 6: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/6.jpg)
The spectral data cube
• E.g. in satellite imaging
– Pushbroom scanning
• Spatial Scanning (2D sensor)
![Page 7: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/7.jpg)
Spatial Scanning• Generalized Mosaics [Schechner & Nayar]
• linear filter
• each pixel column filtered differently
• rotational motion & registration to assemble image stack
![Page 8: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/8.jpg)
• Spectral scanning
The spectral data cube
![Page 9: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/9.jpg)
Frequency Scanning
• Michelson Interferometer with moving mirror - Fourier Transform Imaging Spectroscopy (FTIS)
![Page 10: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/10.jpg)
Imaging Spectrometers
The quest for the instantaneous spectral data cube
(4D Imaging)
![Page 11: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/11.jpg)
Multiplexing - Image slicers
[WiFeS – Wide Field Spectrometer]
![Page 12: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/12.jpg)
[SPIFFI - SPectrometer for Infrared Faint Field Imaging]
Multiplexing - Image slicers
![Page 13: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/13.jpg)
Fiber Optical cables (OKSI)
2D -> 1D reformatting
300 “pixels”
![Page 14: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/14.jpg)
Multiplexing: Prism-Mask Based System
[Du’09]
![Page 15: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/15.jpg)
Computational Imaging Spectrometers
The quest for the instantaneous spectral data cube
![Page 16: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/16.jpg)
• Original method [Descour’95]
• Image a full diffraction pattern
• Perform “CT”
CTIS – Computed Tomography Imaging Spectrometry
diffraction patternspectral image
![Page 17: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/17.jpg)
CTIS in graphics
• HDR imaging
for CTIS(not snapshot due to HDR exposure stack)
[Habel’12]
![Page 18: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/18.jpg)
CTIS in graphics
• Spatial resolution 124x124, 54 bands
[Habel’12]
![Page 19: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/19.jpg)
CASSI – Coded Aperture Snapshot Spectral
Imaging
![Page 20: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/20.jpg)
CASSI – Coded Aperture Snapshot Spectral
Imaging
Implementation with prisms
![Page 21: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/21.jpg)
• Resolution: spatial ~200 x 200 pixels
spectral ~30 bands
CASSI – Coded Aperture Snapshot Spectral Imaging
projection reconstruction (stack) spectra
![Page 22: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/22.jpg)
• Transfer low-res spectra to high res RGB image [Rump’10,Cao’11]
Spectral Transfer
![Page 23: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/23.jpg)
Applications
![Page 24: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/24.jpg)
Applications
• automatic white balancingSpatially uniform illumination
[Cao11]Spatially varying illumination
raw from RGB tungsten WB `greyworld WB spectral WB spectra
![Page 25: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/25.jpg)
Applications
• improved tracking
• real and fake skin detection[Cao11]
RGB – tracking lost
spectral – tracking OK
![Page 26: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/26.jpg)
Applications• analyze / restore paintings
[Calit]
![Page 27: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/27.jpg)
Applications• Satellite-Based Remote Sensing
[DigitalGlobe’10]
vegetation mapping urban land use pollution monitoring
![Page 28: SIGGRAPH 2012 Computational Plenoptic Imaging Course - 3 Spectral Imaging](https://reader036.vdocument.in/reader036/viewer/2022062513/5559f623d8b42aa8098b4860/html5/thumbnails/28.jpg)
• Multispectral at Siggraph’12
– Kim, Harvey, Kittle, Rushmeier, Dorsey, O’Prum, and Brady “3D Imaging Spectroscopy for Measuring Hyperspectral Patterns on Solid Objects”, Monday, 3:45 - 5:35 – “Appearance”
– Hosek and Wilkie, “An Analytic Model for Full Spectral Sky-Dome Radiance”, Wednesday 3:45-5:35 pm – “Physics and Mathematics
for Light”