high dynamic range from multiple images: which exposures to combine? michael grossberg and shree...
Post on 20-Dec-2015
217 views
TRANSCRIPT
High Dynamic Range from Multiple High Dynamic Range from Multiple Images: Images: Which Exposures to Combine?Which Exposures to Combine?
Michael Grossberg and Shree NayarMichael Grossberg and Shree Nayar
CAVE Lab, Columbia UniversityCAVE Lab, Columbia University
ICCV Workshop on CPMCV ICCV Workshop on CPMCV October, 2003, Nice, FranceOctober, 2003, Nice, France
Partially funded by NSF ITR Award, DARPA/ONR MURIPartially funded by NSF ITR Award, DARPA/ONR MURI
Combining Different ExposuresCombining Different Exposures
[Ginosar and Zeevi, 88, Madden, 93,Mann and Picard, 95, Debevec and Malik, 97,Mitsunaga and Nayar, 99]
Low
Dyn
amic
Ran
ge E
xpos
ures
+
Combination Yields High Dynamic Range
The Camera The Camera ResponseResponse
Scene Radiance
Linear Function(Optical Attenuation)
Image Irradiance
L s E
0 255
Camera Response
ImageIntensit
y
Bf
From Response To Measured Irradiance From Response To Measured Irradiance LevelsLevels
Measured Irradiance Levels
Irradiance E
Bri
ghtn
ess
Levels
B
Response Function f
Where do you want your bits?Where do you want your bits?
High Dynamic Range
Coarse quantization
Low Dynamic Range
Fine quantization
Bri
ghtn
ess
BB
rightn
ess
B
Irradiance E
Response Function f
Irradiance E
Response Function f
Effective Camera from Multiple Effective Camera from Multiple ExposuresExposures
CaptureHighDynamic Range
CaptureIrradianceUniformly +
+
Acquired Images Image from Effective CameraGoal
Flexible Dynamic Range Imaging:Flexible Dynamic Range Imaging:
• Can we create an effective camera with a Can we create an effective camera with a desired response?desired response?
– How many exposures are needed?How many exposures are needed?
– Which exposures to acquire? Which exposures to acquire?
– How to combine the acquired images?How to combine the acquired images?
Irradiance Levels From Multiple Irradiance Levels From Multiple ExposuresExposures
f(E)
Irradiance E
E10 E2 E3
21
3
0
2
01
3
6
45
h(E)
f(e2E)
E3 / e3 E2 / e2 E1 / e1 0
21
3
0
Sum
E1
^E6
^E5
^E4
^E3
^E2
^0
Exposure e1 = 1
Effective Camera
Exposure e2
Response of the Effective CameraResponse of the Effective Camera
1
)()(j
EfEh
Number of exposures
Exposures
Camera Response
Irradiance
Effective Response
Theorem: The sum of a set of images of a scene taken at different exposures includes all the information in the individual exposures.
jen
Camera Response EmulationCamera Response Emulation
• How can we tell if How can we tell if hh emulates emulates gg well? well?
Naïve answer:Naïve answer: | h – g | < | h – g | <
gf , ee = ( = (ee11, … ,, … ,eenn))
Level spacing characterizes similarity
Bri
ghtn
ess
Levels
B
Irradiance E
h
Desired Response
Emulated Responsedepends on:
h~
How Response Determines Level How Response Determines Level SpacingSpacing
Measured Irradiance Levels
Irradiance E
Bri
ghtn
ess
Levels
B
h
Observation: The derivative determines the distances between levels.
Dense Spacing
LargerDerivative
Sparse Spacing
SmallerDerivatives
The Objective FunctionThe Objective Function
MAX
MIN
E
E
dEwhg 2||),(
Spacing Based Comparison:
Weight prevents penalizing success:
Desired Response Effective Response
Weight
Number of Exposures Exposure Values
en
)(Ew { )()( EhEg 0,
1, otherwise
B
rightn
ess
B
g
Irradiance E
h
w=0 w=1
Which Exposures and How Many?Which Exposures and How Many?
• For fixed For fixed nn, find minimizing exposure , find minimizing exposure values values ee
• Choose min Choose min nn such that error within such that error within tolerance tolerance
• Method: Exhaustive searchMethod: Exhaustive search
– Objective function not continuousObjective function not continuous– Only need to search actual settings Only need to search actual settings – Offline build table of exposuresOffline build table of exposures
Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging
Linear
Gamma =1/2
Constantcontrast(log)
1111111111111111
21.0031.0031.0151.0031.0033.0941.0031.0031.0031.00320.249.914.6894.8313.979
3
2.9851.0191.0071.007
5.1463.0191.0061.006
88.3837.2329.1816.01
4
3.6721.1661.031
11.235.0492.866
280144.964.22
5
4.9751.078
18.568.564
763.5305.4
6
5.636
33.7
1130
EffectiveCamera
Number ofExposures
Exposure Values
234562345623456
Desired Response
Constant Contrast
Gamma 1/2
Linear
High 1:16,000
Low 1:256
Dynamic Range
Medium 1:1,000
Desired Response
Constant Contrast
Gamma 1/2
Linear
High 1:16,000
Low 1:256
Dynamic Range
Linear
Gamma =1/2
Constantcontrast(log)
1111111111111111
21.0031.0031.0151.0031.0033.0941.0031.0031.0031.00320.249.91
4.6894.8313.979
3
2.9851.0191.0071.007
5.1463.0191.0061.006
88.3837.2329.1816.01
4
3.6721.1661.031
11.235.0492.866
280144.964.22
5
4.9751.078
18.568.564
763.5305.4
6
5.636
33.7
1130
EffectiveCamera
Number ofExposures
Exposure Values
2345623456
23456
Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging
Baseline Exposure ValuesBaseline Exposure Values
• Typically exposures are doubledTypically exposures are doubled
• Baseline: Baseline: Combine the exposures Combine the exposures e e =(1,2,4)=(1,2,4)
[Ginosar and Zeevi, 88, Madden, 93,Mann and Picard, 95, Debevec and Malik, 97,Mitsunaga and Nayar, 99]
Increased Dynamic Range Linear Increased Dynamic Range Linear CameraCamera
• Real Camera: Real Camera: f f linear linear • Desired Camera: Desired Camera: gg linear (greater dynamic linear (greater dynamic
range) range)
Baseline Response (1,2,4)
Desired Response
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0
Irradiance E
Bri
ghtn
ess
B
4-bit real camera
Baseline Response (1,2,4)
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0
Desired Response
Irradiance E
Bri
ghtn
ess
B
8-bit real camera
Computed Response(1, 1.003, 2.985)
Computed Response(1, 1.05, 1.11)
Linear Camera: Synthetic Ramp Linear Camera: Synthetic Ramp ImageImage
From computed exposures (1,1.05,1.11)From baseline exposures (1,2,4)
High Dynamic Range Linear Camera
Linear Camera: Image of ClothLinear Camera: Image of Cloth
Computed Exposures (1,1.05,1.11)
Ground Truth (HDR image)
Baseline Exposures (1,2,4)
Constant Contrast from Linear Constant Contrast from Linear CamerasCameras
• Real Camera: Real Camera: f f linear linear • Desired Camera: Desired Camera: gg log response (constant log response (constant
contrast)contrast)
Baseline Response (1,2,4)
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0
Irradiance E
Brig
htne
ss B
Desired Response
3 Exposures
Baseline Response
Irradiance E
1.0
0.8
0.6
0.4
0.2
0.00.80.60.40.20.0 1.0
Brig
htne
ss B
Desired Response
5 Exposures
Computed Response (1,9.91,88.38)
ComputedResponse
Constant Contrast : Image of TilesConstant Contrast : Image of Tiles
Computed Exposures
Baseline Exposures
Linear Camera from Non-linear CameraLinear Camera from Non-linear Camera
Baseline Input Exposures
Iso-brightnessCombined Iso-brightness
Computed Input Exposures
Combined
• Real Camera: Real Camera: f f non-linear (Nikon 990) non-linear (Nikon 990)• Desired Camera: Desired Camera: gg linear linear
SummarySummary
• Combine images using Combine images using summationsummation
• Method finds Method finds number of exposuresnumber of exposures and and exposure valuesexposure values to use to use
• Emulation of a variety of cameras:Emulation of a variety of cameras:
Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging