hyperspectral image projector (hip) - nasa
TRANSCRIPT
Page 1 HIP Tour Briefing
Hyperspectral Image Projector (HIP)
•A scene projector for in-lab testing of imaging sensors
using spectrally-realistic scenes
Additional information and
publications available from:
Or go to the NIST website ( www.nist.gov ) and search “HIP”
Page 2 HIP Tour Briefing
Laboratory sources do not match reality very closely
We calibrate with uniform sources…
Example: lamp-illuminated
integrating sphere for reflective bands,
(or blackbody for IR emissive bands)
But reality is spatially non-uniform:
Example: AVIRIS image of
North Island Naval Air Station,
San Diego, CA
Page 3 HIP Tour Briefing
The same situation applies spectrally
0.0000
50.000
100.00
150.00
200.00
250.00
400 450 500 550 600 650 700 750
Ra
dia
nc
e
Wavelength [nm]
Lamps standards and blackbodies offer
only a Planckian-shaped spectrum.
But reality has many different spectra…
Example: ENVI/SMACC was used to find
these 7 endmember spectra from the
San Diego Naval Air Station data cube.
NIST FEL Lamp
0
2000
4000
6000
8000
10000
12000
450 500 550 600 650 700 750
Fig5
EM1EM2EM3EM4
EM5EM6EM7
Inte
nsi
ty (
arb
itra
ry u
nits
)Wavelength (nm)
SMACC Reference: J. Gruninger, A. J. Ratkowski, and M. L. Hoke,
“The sequential maximum angle convex cone (SMACC) endmember model,”
Proc. SPIE 5425, 1-14 (2004).
Page 4 HIP Tour Briefing
0
100
200
300
400
500
600
400 500 600 700 800 900 1000 1100
OLI Matching Absolute
Sp
ectr
al R
adia
nce
(W
/m2sr
um
)
Wavelength (nm)
TOA Solar
Bare Desert Soil
Vegetation
HIP VNIR Limits
Measured from HIP Prototype VNIR Spectral Engine 11/3/10
Modeled Spatial Engine: XGA DMD f/3 with 20% Transmittance
The Hyperspectral Image Projector (HIP) Can Match Typical
Reflected-Solar Radiance Spectra
• The HIP provides enough light to simulate a bright sunny day outside
• Red data plots below show how well the HIP simulates different real-world spectra
Page 5 HIP Tour Briefing
• Broadband Fiber Light Source
– Provides high brightness in a small spot for high spectral resolution
– Fused-silica fiber supercontinuum source for VNIR/SWIR
• Spectral Engine
– Fiber source light is spectrally dispersed across a digital micromirror device (DMD)
– Image reflected from on-pixels of DMD is spatially integrated and determines spectrum
• Spatial Engine
– Uses another DMD to project spectra into customer’s Unit Under Test (UUT) per 2D spatial
abundance images, through a collimator, in sync with spectral engine
• Reference Instrument
– Well-calibrated imaging spectrometer to measure the spectrum of each spatial pixel at the
output, relative to NIST reference standards, to provide truth data
HIP Prototype OverviewSpectral Engine
(Spectra Generator)
Reference InstrumentBroadband Fiber
Light Source
Spatial Engine
(Image Projector) Unit Under Test
Computer
Page 6 HIP Tour Briefing
Supercontinuum Fiber Laser: A “White” Broadband Laser
• Utilizes non-linear effects in a photonic crystal optical fiber to greatly broaden the spectrum of a 1064 nm pump laser.
• Broadband light is generated in a single-mode (5 um core diameter) photonic crystal (holey) optical fiber
– No etendue issues as with lamps or blackbodies.
– Ideally suited for coupling to a spectral engine.
• High power and high spectral resolution:
– 3mW/nm spectral power density from 450 nm to 1700 nm
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
500 1000 1500 2000 2500
SC450 ASD
Powe
r (ar
bitra
ry u
nits
)
Wavelength (nm)
Page 7 HIP Tour Briefing
Digital Micromirror Device (DMD)
• An array of MEMS micromirror elements
• Developed by Texas Instruments (TI)
– 1024 x 768 elements, +/- 12 degree tilt angle
– Aluminum mirrors
– 13.7 micron pitch
– < 24 microseconds mechanical switching time.
• For visible to 2500 nm applications we have used:
TI Discovery 1100 electronics board with an
Accessory Light Processor (ALP) electronics board
(see dlp.com).
• For longer wavelength infrared developments we are using DMDs where the glass window is replaced by a ZnSe window.
• Control algorithms are being written by us using LabVIEW with a USB interface to a standard PC.
• For the prototypes in this proposal, we plan to use the TI Discovery 3000 and ALP3.
MEMS = Micro-Electro-Mechanical System
MAPS = Micromirror Array Projection System
Page 8 HIP Tour Briefing
Principle of the Spectrally Programmable Source
(also called the Spectral Engine)
Dispersing element Spatial Light Modulator(SLM) Recombine the Light
SLM Light Guide
• Enabling Technology: SLM
– Digital micromirror devices (DMDs)
– Liquid crystal on silicon arrays (LCOS arrays)
Page 9 HIP Tour Briefing
How the DMD is used in a spectrally tunable source:
Monochromator Mode
Near IRUV Visible
1024 mirrors
768
mir
rors
Wavelength
Inte
nsi
ty
mirror array
Wavelength and Intensity Selection with a Digital Mirror Device (note mirror # reduced by a factor of 10 for clarity)
Mirrors in "On"Position
Page 10 HIP Tour Briefing
Near IRUV Visible
Digital Mirror Device as a Spectral Light Engine (note mirror # reduced by a factor of 10 for clarity)
1024 mirrors
768
mir
rors
Wavelength
Inte
nsi
ty
mirror array
How the DMD is used to create an arbitrarily programmable
spectrum
Page 11 HIP Tour Briefing
Design Concept for
VNIR/SWIR Spectral Engine
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
500 1000 1500 2000 2500
SC450 ASD
Powe
r (ar
bitra
ry u
nits
)
Wavelength (nm)
Supercontinuum
Fiber Light Source
DMD2
Recombining
System
Dispersing Prisms
DMD1
VNIR SWIR
Input
Output to
Spatial
Engine
SWIR
VNIR
• Splits input into VNIR and SWIR
• One DMD (1024x768) for each range
• Optical modeling in Zemax and FRED
– Prisms selected
– Design meets requirements
❖Spectral range
❖Transmittance
❖Spectral resolution
• Mechanical modeling in Solid Works
VNIR/SWIR spectral
engine fits in 19” rack-
mountable chassis
Source Spectrum
Page 12 HIP Tour Briefing
HIP VNIR-SWIR Spectral Engine
Inside view from back:Outside view from front:
Supercontinuum Light Source
VNIR
Spectral
Engine
SWIR
Spectral
Engine
Page 13 HIP Tour Briefing
HIP VNIR-SWIR Spectral Engine
Supercontinuum
Light Source
VNIR
Spectral
Engine
SWIR
Spectral
Engine
Source
Collimator
Electronics
Section
Output
}}
View from back:
Dichroic
Beamsplitter
Dichroic
Beamcombiner
Page 14 HIP Tour Briefing
VNIR
Spectral
Engine
SWIR
Spectral
Engine
Spatial
Engine
Collimator For Projection
to UUT
Super-
continuum
Source
Power Supply
Page 15 HIP Tour Briefing
0
100
200
300
400
500
600
400 500 600 700 800 900 1000 1100
OLI Matching Absolute
Sp
ectr
al R
adia
nce
(W
/m2sr
um
)
Wavelength (nm)
TOA Solar
Bare Desert Soil
Vegetation
HIP VNIR Limits
Measured from HIP Prototype VNIR Spectral Engine 11/3/10
Modeled Spatial Engine: XGA DMD f/3 with 20% Transmittance
The Hyperspectral Image Projector (HIP) Can Match Typical
Reflected-Solar Radiance Spectra
• The HIP provides enough light to simulate a bright sunny day outside
• Red data plots below show how well the HIP simulates different real-world spectra
Page 16 HIP Tour Briefing
HIP Prototype Specifications
Parameter Specification
Spectral Range450 nm to 2500 nm (VNIR-SWIR)
(extension to 350 nm in progress)
Spectral Resolution5 nm VNIR
8 nm SWIR
VNIR/SWIR/MWIR Sync. Accuracy 1 microsecond
Spatial Format 1024 H × 768 V
Projected FOV** 7.9° H × 5.9° V
Spatial Resolution** 0.135 mrad
Average Spectral Radiance 1000 W/m2srmm
Bit Depth and Frame Rate
12 bits at 250 Hz max;
8 bits per component at180 Hz/N typical*
1 Bit at 11 kHz max
Contrast Ratio 1000:1
Wavelength Accuracy 2 nm
Radiance Accuracy 2%
*N = number of components (i.e. eigenspectra) per frame
**Depends on collimator used. Values shown are for the standard 100 mm collimator. 500 mm collimator also available.
Page 17 HIP Tour Briefing
VNIR Spectral Resolution Test Data
0
0.2
0.4
0.6
0.8
1
628 628.5 629 629.5 630 630.5 631 631.5 632
Single Column 629 nm HIP HR4000 18Oct2010
Respo
nse (
peak-n
orm
aliz
ed
)
Wavelength (nm)
1.5 nmMeets the < 5 nm spec
Spectral Resolution of Test
Instrumentation is 0.2 nm
Page 18 HIP Tour Briefing
SWIR Spectral Resolution Test Data
Meets the < 8 nm spec
Spectral Resolution of Test
Instrumentation is 0.5 nm
0.0
0.2
0.4
0.6
0.8
1.0
1585 1590 1595 1600 1605 1610
IB 1600 nm HIP Slocal 06Jun2011
Re
sp
on
se
(p
ea
k-n
orm
aliz
ed
)
Wavelength (nm)
5 nm
Page 19 HIP Tour Briefing
VNIR-SWIR Spatial Engine
Prototype Mechanical Design
DMD
Interchangeable DMD Driver Electronics PC Boards:
Cooling FansReplaceable
Collimator
Purged, Sealed
Optics SectionLiquid Light Guide
Coupling from
Spectral Engine
Electrical
Connections
to Spectral
Engine
Crossfield/Infiniband (pink)
or
DLi3000/DVI (red)
or
D3000/ALP (green):
Page 20 HIP Tour Briefing
Background: Digital Light Processing (DLP) Projectors
Reference: www.dlp.com
Page 21 HIP Tour Briefing
HIP Image Cube Projection Concept
Spatial Engine
• Projects images with
component spectra into
sensor
• Reference instrument
characterizes output
Reference
Instrument
Sensor
Under Test
Supercontinuum
Light SourceSpectral Engine
• Uses light from supercontinuum source
• Produces programmable spectra that
match component spectra
• Directs these to Spatial Engine
0.0
0.2
0.4
0.6
0.8
1.0
400 450 500 550 600 650 700 750
Pow
er
Wavelength (nm)
Example Spectrum
Input
Hyperspectral
Image CubeComponent
Spectra
Principle-components-type
algorithm reduces image cube
to Component Spectra
and Abundance Images
Abundance
Images
Page 22 HIP Tour Briefing
Input
Image Cube
Example: AVIRIS Image Cube
of San Diego Naval Air Station
First, ENVI/SMACC was used to find these
Endmember Spectra and their Abundances
J. Gruninger, A. J. Ratkowski, and M. L. Hoke, “The sequential maximum angle
convex cone (SMACC) endmember model,” Proc. SPIE 5425, 1-14 (2004).
EM-3
EM-4
EM-2 EM-5
EM-6
Eigenspectra Abundances
EM-1
EM-1
Eigenspectra
EM-2
EM-3
EM-5
EM-4
EM-6
EM-7
Compressive Projection is Used to Achieve Higher Brightness
Then we need only project N = 6 broadband spectra
instead of M = 30+ monochromatic spectra.
Page 23 HIP Tour Briefing
Example Sensor
Test at the HIP
• Used a pushbroom Hyperspectral Imager (HSI) from collaborators at University of
Colorado – This sensor is prototype instrument for NASA.
• Input data was a real scene collected by HSI
• Projected by the HIP and measured by the HSI.
• HSI scanned HIP to simulate ground track motion
HIP Projected, HSI Measured: HSI
Page 24 HIP Tour Briefing
HIP Projection of MODIS Satellite Image Into ORCA Prototype in the Lab
Original Image HIP Image Measured by ORCA Difference Image (at 645 nm)
• ORCA (Ocean Radiometer for Carbon Assessment)
was a NASA prototype ocean color sensor for the
NASA PACE (Pre- Aerosol Cloud Ecosystems) mission.
• Clouds provide stray light that interferes
with the ability to measure ocean color and
bio-chemistry to quantify carbon processes
• HIP difference images potentially allow
measurement of stray light effects with
realistic scenes
Typical Cloud Pixel Typical Ocean Pixel
Page 25 HIP Tour Briefing
Monochromatic Band Projection
•Using the HIP, we projected six MODIS
monochromatic band images into ORCA at the following
band center wavelengths:
“Eigenspectrum #” Wavelength (nm)
1 645
2 667
3 678
4 748
5 859
6 869
•All six images were projected sequentially, but all within
the integration time of ORCA
•The radiance levels at each wavelength had been
adjusted prior to the test, using a calibrated detector at
the output, so that they would have equal radiance
levels for a spatially uniform scene
-0.02
0.00
0.02
0.04
0.06
0.08
0.10
0.12
550 600 650 700 750 800 850 900 950
ASD_111207_002
Ra
dia
nce (
arb
itra
ry u
nits)
Wavelength (nm)
•Measurement of radiance levels of the 6
monochromatic bands from the HIP as
made using an ASD spectrometer:
Page 26 HIP Tour Briefing
ORCA
HIP
Spatial
Engine
Liquid Light Guide:
•Couples HIP Spectral Engine
to HIP Spatial Engine
Translation stage moves HIP between Reference Instruments and ORCA
NIST Reference Instruments:
•Enable measurements of HIP
output radiance (ground truth)
•Includes an ASD spectrometer
and a CCD camera with tunable filters
ORCA at the HIP lab at NIST
Page 27 HIP Tour Briefing
HIP Prototype Specifications
Parameter Specification
Spectral Range450 nm to 2500 nm (VNIR-SWIR)
(extension to 350 nm in progress)
Spectral Resolution5 nm VNIR
8 nm SWIR
VNIR/SWIR/MWIR Sync. Accuracy 1 microsecond
Spatial Format 1024 H × 768 V
Projected FOV** 7.9° H × 5.9° V
Spatial Resolution** 0.135 mrad
Average Spectral Radiance 1000 W/m2srmm
Bit Depth and Frame Rate
12 bits at 250 Hz max;
8 bits per component at180 Hz/N typical*
1 Bit at 11 kHz max
Contrast Ratio 1000:1
Wavelength Accuracy 2 nm
Radiance Accuracy 2%
*N = number of components (i.e. eigenspectra) per frame
**Depends on collimator used. Values shown are for the standard 100 mm collimator. 500 mm collimator also available.