![Page 1: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/1.jpg)
High Dynamic Range from Multiple High Dynamic Range from Multiple Images: Images: Which Exposures to Combine?Which Exposures to Combine?
Michael Grossberg and Shree NayarMichael Grossberg and Shree Nayar
CAVE Lab, Columbia UniversityCAVE Lab, Columbia University
ICCV Workshop on CPMCV ICCV Workshop on CPMCV October, 2003, Nice, FranceOctober, 2003, Nice, France
Partially funded by NSF ITR Award, DARPA/ONR MURIPartially funded by NSF ITR Award, DARPA/ONR MURI
![Page 2: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/2.jpg)
Combining Different ExposuresCombining Different Exposures
[Ginosar and Zeevi, 88, Madden, 93,Mann and Picard, 95, Debevec and Malik, 97,Mitsunaga and Nayar, 99]
Low
Dyn
amic
Ran
ge E
xpos
ures
+
Combination Yields High Dynamic Range
![Page 3: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/3.jpg)
The Camera The Camera ResponseResponse
Scene Radiance
Linear Function(Optical Attenuation)
Image Irradiance
L s E
0 255
Camera Response
ImageIntensit
yBf
![Page 4: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/4.jpg)
From Response To Measured Irradiance From Response To Measured Irradiance LevelsLevels
Measured Irradiance Levels
Irradiance E
Brig
htne
ss L
evel
s B
Response Function f
![Page 5: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/5.jpg)
Where do you want your bits?Where do you want your bits?
High Dynamic Range
Coarse quantization
Low Dynamic Range
Fine quantization
Brig
htne
ss B
Brig
htne
ss B
Irradiance E
Response Function f
Irradiance E
Response Function f
![Page 6: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/6.jpg)
Effective Camera from Multiple Effective Camera from Multiple ExposuresExposures
CaptureHighDynamic Range
CaptureIrradianceUniformly +
+
Acquired Images Image from Effective CameraGoal
![Page 7: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/7.jpg)
Flexible Dynamic Range Imaging:Flexible Dynamic Range Imaging:
• Can we create an effective camera with a Can we create an effective camera with a desired response?desired response?
– How many exposures are needed?How many exposures are needed?
– Which exposures to acquire? Which exposures to acquire?
– How to combine the acquired images?How to combine the acquired images?
![Page 8: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/8.jpg)
Irradiance Levels From Multiple Irradiance Levels From Multiple ExposuresExposures
f(E)
Irradiance E
E10 E2 E3
213
0
2013
645
h(E)
f(e2E)
E3 / e3 E2 / e2 E1 / e1 0
213
0
Sum
E1^ E6
^E5^E4
^E3^E2
^0
Exposure e1 = 1
Effective Camera
Exposure e2
![Page 9: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/9.jpg)
Response of the Effective CameraResponse of the Effective Camera
1
)()(j
EfEh
Number of exposures
Exposures
Camera Response
Irradiance
Effective Response
Theorem: The sum of a set of images of a scene taken at different exposures includes all the information in the individual exposures.
jen
![Page 10: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/10.jpg)
Camera Response EmulationCamera Response Emulation
• How can we tell if How can we tell if hh emulates emulates gg well? well? Naïve answer:Naïve answer: | h – g | < | h – g | <
gf , ee = ( = (ee11, … ,, … ,eenn))
Level spacing characterizes similarity
Brig
htne
ss L
evel
s B
Irradiance E
h
Desired Response
Emulated Responsedepends on:
h~
![Page 11: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/11.jpg)
How Response Determines Level How Response Determines Level SpacingSpacing
Measured Irradiance Levels
Irradiance E
Brig
htne
ss L
evel
s B
h
Observation: The derivative determines the distances between levels.
Dense Spacing
LargerDerivative
Sparse Spacing
SmallerDerivatives
![Page 12: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/12.jpg)
The Objective FunctionThe Objective Function
MAX
MIN
E
E
dEwhg 2||),(
Spacing Based Comparison:
Weight prevents penalizing success:
Desired Response Effective Response
Weight
Number of Exposures Exposure Values
en
)(Ew { )()( EhEg 0,1, otherwise
B
right
ness
B
g
Irradiance E
h
w=0 w=1
![Page 13: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/13.jpg)
Which Exposures and How Many?Which Exposures and How Many?• For fixed For fixed nn, find minimizing exposure , find minimizing exposure
values values ee
• Choose min Choose min nn such that error within such that error within tolerance tolerance
• Method: Exhaustive searchMethod: Exhaustive search
– Objective function not continuousObjective function not continuous– Only need to search actual settings Only need to search actual settings – Offline build table of exposuresOffline build table of exposures
![Page 14: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/14.jpg)
Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging
Linear
Gamma =1/2
Constantcontrast(log)
1111111111111111
21.0031.0031.0151.0031.0033.0941.0031.0031.0031.00320.249.914.6894.8313.979
3
2.9851.0191.0071.007
5.1463.0191.0061.006
88.3837.2329.1816.01
4
3.6721.1661.031
11.235.0492.866
280144.964.22
5
4.9751.078
18.568.564
763.5305.4
6
5.636
33.7
1130
EffectiveCamera
Number ofExposures
Exposure Values
234562345623456
![Page 15: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/15.jpg)
Desired ResponseConstant Contrast
Gamma 1/2Linear
High 1:16,000
Low 1:256
Dynamic Range
Medium 1:1,000
Desired ResponseConstant Contrast
Gamma 1/2Linear
High 1:16,000
Low 1:256
Dynamic Range
Linear
Gamma =1/2
Constantcontrast(log)
1111111111111111
21.0031.0031.0151.0031.0033.0941.0031.0031.0031.00320.249.91
4.6894.8313.979
3
2.9851.0191.0071.007
5.1463.0191.0061.006
88.3837.2329.1816.01
4
3.6721.1661.031
11.235.0492.866
280144.964.22
5
4.9751.078
18.568.564
763.5305.4
6
5.636
33.7
1130
EffectiveCamera
Number ofExposures
Exposure Values
234562345623456
Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging
![Page 16: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/16.jpg)
Baseline Exposure ValuesBaseline Exposure Values
• Typically exposures are doubledTypically exposures are doubled
• Baseline: Baseline: Combine the exposures Combine the exposures e e =(1,2,4)=(1,2,4)
[Ginosar and Zeevi, 88, Madden, 93,Mann and Picard, 95, Debevec and Malik, 97,Mitsunaga and Nayar, 99]
![Page 17: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/17.jpg)
Increased Dynamic Range Linear Increased Dynamic Range Linear CameraCamera• Real Camera: Real Camera: f f linear linear • Desired Camera: Desired Camera: gg linear (greater dynamic linear (greater dynamic
range) range) Baseline Response (1,2,4)
Desired Response
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0
Irradiance E
Brig
htne
ss B
4-bit real camera
Baseline Response (1,2,4)
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0
Desired Response
Irradiance E
Brig
htne
ss B
8-bit real camera
Computed Response(1, 1.003, 2.985)
Computed Response(1, 1.05, 1.11)
![Page 18: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/18.jpg)
Linear Camera: Synthetic Ramp Linear Camera: Synthetic Ramp ImageImage
From computed exposures (1,1.05,1.11)From baseline exposures (1,2,4)
High Dynamic Range Linear Camera
![Page 19: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/19.jpg)
Linear Camera: Image of ClothLinear Camera: Image of Cloth
Computed Exposures (1,1.05,1.11)
Ground Truth (HDR image)
Baseline Exposures (1,2,4)
![Page 20: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/20.jpg)
Constant Contrast from Linear Constant Contrast from Linear CamerasCameras• Real Camera: Real Camera: f f linear linear • Desired Camera: Desired Camera: gg log response (constant log response (constant
contrast)contrast)
Baseline Response (1,2,4)
0.8
0.6
0.4
0.2
0.0
1.0
0.80.60.40.20.0 1.0Irradiance E
Brig
htne
ss B
Desired Response
3 Exposures
Baseline Response
Irradiance E
1.0
0.8
0.6
0.4
0.2
0.00.80.60.40.20.0 1.0
Brig
htne
ss B
Desired Response
5 Exposures
Computed Response (1,9.91,88.38)
ComputedResponse
![Page 21: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/21.jpg)
Constant Contrast : Image of TilesConstant Contrast : Image of Tiles
Computed Exposures
Baseline Exposures
![Page 22: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/22.jpg)
Linear Camera from Non-linear CameraLinear Camera from Non-linear Camera
Baseline Input Exposures
Iso-brightnessCombined Iso-brightness
Computed Input Exposures
Combined
• Real Camera: Real Camera: f f non-linear (Nikon 990) non-linear (Nikon 990)• Desired Camera: Desired Camera: gg linear linear
![Page 23: High Dynamic Range from Multiple Images: Which Exposures to Combine?](https://reader036.vdocument.in/reader036/viewer/2022070421/568160c7550346895dcff72c/html5/thumbnails/23.jpg)
SummarySummary• Combine images using Combine images using summationsummation
• Method finds Method finds number of exposuresnumber of exposures and and exposure valuesexposure values to use to use
• Emulation of a variety of cameras:Emulation of a variety of cameras: Flexible Dynamic Range ImagingFlexible Dynamic Range Imaging