burst photography - stanford university · computational photography - overview! debevec &...
TRANSCRIPT
![Page 1: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/1.jpg)
Burst Photography!
Gordon Wetzstein!Stanford University!
EE367/CS448I: Computational Imaging and Display!stanford.edu/class/ee367!
Lecture 7!
![Page 2: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/2.jpg)
Motivation!
exposure sequence!
Motivation
exposure sequence
wik
iped
ia!
-4 stops!
![Page 3: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/3.jpg)
Motivation!
exposure sequence!
wik
iped
ia!
-2 stops!
![Page 4: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/4.jpg)
Motivation!
exposure sequence!
wik
iped
ia!
2 stops!
![Page 5: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/5.jpg)
wik
iped
ia!
4 stops!
![Page 6: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/6.jpg)
Motivation!w
ikip
edia!
HDR!contrast
reduction (scaling)!
![Page 7: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/7.jpg)
wik
iped
ia!
HDR!local tone mapping!
![Page 8: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/8.jpg)
Computational Photography - Overview!
Deb
evec
& M
alik
, 199
7!• high dynamic range!• super-resolution!
• burst photography!
• focal stack!
• aperture stack!
• confocal stereo!• blurry/noisy!
• flash/no flash!
• multi-flash!
![Page 9: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/9.jpg)
High Dynamic Range Imaging!
•! dynamic range: ratio between brightest and darkest value!•! quantization within that range is equally important !
! from 8 bits (256 values) to 32 bits floating point!
original photo! motion blurred photo!simulation from HDR!simulation from LDR!
Deb
evec
& M
alik
, 199
7!
![Page 10: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/10.jpg)
HDRI – Overview!
• estimate camera response curve!• capture multiple low dynamic range (LDR) exposures!
• fuse LDR images into 32 bit HDR image!
• possibly convert to absolute radiance (global scaling)!
• application specific use:!
• image-based rendering lighting!• tone mapping !
• …!
![Page 11: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/11.jpg)
HDRI – Estimating the Response Curve!
• not required when working with linear RAW images !• easiest option: use calibration chart!
![Page 12: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/12.jpg)
HDRI – Estimating the Response Curve!
•! not required when working with linear RAW images !•! easiest option: use calibration chart!
0pixel value!
128! 255!64! 196!kn
own
refle
ctan
ce!
1
linear RAW!
![Page 13: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/13.jpg)
HDRI – Estimating the Response Curve!
•! not required when working with linear RAW images !•! easiest option: use calibration chart!
0pixel value!
128! 255!64! 196!kn
own
refle
ctan
ce!
1
e.g. JPEG!
12864 196
![Page 14: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/14.jpg)
HDRI – Linearizing LDR Exposures!
•! capture exposure, apply lookup table!
0pixel value!
128! 255!64! 196!re
lativ
e ra
dian
ce!
1
e.g. JPEG!
12864 196
I
Ilin = f !1 I( )
f !1 "( )
![Page 15: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/15.jpg)
HDRI – Merging LDR Exposures!
!Image from Debevec & Malik, 1997!
• start with LDR image sequence Ii (only exposure time ti changes)!• individual exposure is: , f is camera response function!Ii = f tiX( )
![Page 16: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/16.jpg)
HDRI – Merging LDR Exposures!
!Image from Debevec & Malik, 1997!
• undo the camera response:!e.g. gamma function!
Ilini = f −1 Ii( )f I( ) = I1/γ → f −1 I( ) = I γ
![Page 17: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/17.jpg)
HDRI – Merging LDR Exposures!
•! compute a weight (confidence) that a pixel is well-exposed !! (close to) saturated pixel = not confident, pixel in center of dynamic
range = confident!!
! wij = exp !4Ilinij ! 0.5( )20.52
"
#
$$
%
&
''
or mean pixel value,!e.g. 127.5 if I in [0, 255]!
&
''&'&
'''
![Page 18: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/18.jpg)
HDRI – Merging LDR Exposures!wij = exp −4
Ilinij − 0.5( )20.52
⎛
⎝
⎜⎜
⎞
⎠
⎟⎟
• compute per-color-channel-per-LDR-pixel weights!
![Page 19: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/19.jpg)
HDRI – Merging LDR Exposures!
• define least-squares objective function in log-space à perceptually linear:!
• equate gradient to zero:!
• gives:!
minimize O=X
wi log Ilini( )− log tiX( )( )i∑2
∂O∂log X( ) = 2 wi log Ilini( )− log ti( )− log X( )( )i∑ = 0
X! = expwi log Ilini( )− log ti( )( )i∑
wii∑⎛
⎝⎜⎜
⎞
⎠⎟⎟
![Page 20: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/20.jpg)
HDRI – Merging LDR Exposures!
•! define least-squares objective function in log-space ! perceptually linear:!
•! equate gradient to zero:!
•! gives:!
minimize O=X
wi log Ilini( )! log tiX( )( )i"2
!O!log X( ) = 2 wi log Ilini( )" log ti( )" log X( )( )i# = 0
X! = expwi log Ilini( )! log ti( )( )i"
wii"#
$%%
&
'((
X! = expwi log Ilini( )! log ti( )( )i"
wii"#
$%#%#
%$%$%%%
&
'(&(&
('('(((
![Page 21: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/21.jpg)
HDRI – Relative v Absolute Radiance!
•! LDR to HDR only gives relative radiance (HW4!)!•! scale by reference radiance to get absolute!!
!Im
age
from
Deb
evec
& M
alik
, 199
7!
![Page 22: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/22.jpg)
•! text!• text
Image-based Lighting with Light Probes!
Paul Debevec!
![Page 23: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/23.jpg)
•! single light probe covers light incident from (almost) entire hemisphere!!
Image-based Lighting with Light Probes!
![Page 24: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/24.jpg)
Paul
Deb
evec
, Ren
derig
n w
ith N
atur
al L
ight!
SIG
GRA
PH E
lect
roni
c Th
eate
r 199
8!
Imag
e Ba
sed
Ligh
ting!
![Page 25: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/25.jpg)
HDRI – Tone Mapping!
• how to display a high dynamic range image on an LDR display? !
• tone mapping: fit into luminance range of display (or 0-255), while
preserving image details !
• HW4!
![Page 26: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/26.jpg)
HDRI – Tone Mapping!
[Dur
and
and
Dor
sey,
200
2]!
• sun overexposed!• foreground too dark!
![Page 27: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/27.jpg)
HDRI – Global Tone Mapping!
[Dur
and
and
Dor
sey,
200
2]!
• gamma correction:!
• colors are washed out!
I = I γ
![Page 28: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/28.jpg)
HDRI – Global Tone Mapping!
[Dur
and
and
Dor
sey,
200
2]!
• gamma in intensity only!!
• intensity details lost !
![Page 29: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/29.jpg)
HDRI – Gradient-domain Tone Mapping!
•! compute gradients, scale them, integrate (Poisson eq.) !
[Fat
tal e
t al.,
200
2]!
[Fat
tal e
t al.,
200
2]
HDR image (scaled)!
![Page 30: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/30.jpg)
HDRI – Gradient-domain Tone Mapping!
•! compute gradients, scale them, integrate (Poisson eq.) !
[Fat
tal e
t al.,
200
2]!
[Fat
tal e
t al.,
200
2]
HDR image (scaled)!gradients!
![Page 31: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/31.jpg)
HDRI – Gradient-domain Tone Mapping!
tone mapped result!gradient attenuation map!
HDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone MappingHDRI – Gradient-domain Tone Mapping
[Fat
tal e
t al.,
200
2]!
![Page 32: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/32.jpg)
Fast!Bilateral !Filter!
HDRI – Tone Mapping with Bilateral Filter!
Detail!Color!
Intensity! Large scale (base layer)!
Reduce!contrast!
Detail!
Large scale!
Color!
Preserve!!
Input HDR image!
Output!
[Dur
and
and
Dor
sey,
200
2]!
![Page 33: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/33.jpg)
HDRI – Tone Mapping with Bilateral Filter!
[Dur
and
and
Dor
sey,
200
2]!
Gradient-space [Fattal et al.]! Bilateral [Durand et al.]!
•! difference is not too big!
![Page 34: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/34.jpg)
HDRI – Tone Mapping with Bilateral Filter!
[Dur
and
and
Dor
sey,
200
2]!
Gradient-space [Fattal et al.]! Bilateral [Durand et al.]!
•! bilateral “looks” a bit better!
•! no ground truth ! it’s up
to the user!
![Page 35: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/35.jpg)
HW4, Q1 & Q2!
• Q1: HDR image fusion (from series of different LDR exposures)!
• Q2: tone-map HDR image with!
• global gamma correction on all color channels!
• global gamma correction on intensity channel!
• local tone mapping with bilateral filter!
![Page 36: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/36.jpg)
Burst Photography - Overview!
•! basic idea: capture and merge bursts of photos (2 or more):!•! multiple exposures: HDR but also deblurring …!
•! multiple shifted low-res images: super-resolution!
•! focal stack!
•! aperture stack!
•! noisy + blurry: denoising + deblurring!•! flash / no flash!
•! multi-flash!
• multiple exposures: HDR but also deblurring …
![Page 37: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/37.jpg)
Pixel Super-Resolution!
• increase “pixel count”, not related to diffraction limit!• idea: capture multiple low-res (LR) images and fuse them into a single
super-resolved (SR) image!
Super-Resolution!
[Ben
-Ezr
a et
al.,
200
4]!
![Page 38: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/38.jpg)
Pixel Super-Resolution!lig
ht l1
6!
![Page 39: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/39.jpg)
light
l16!
![Page 40: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/40.jpg)
Pixel Super-Resolution!
•! LR must be sub-pixel shifted!
I1
I2
ISR
I1I2
!
"##
$
%&&=
A1A2
!
"##
$
%&&ISR
stacked, measured!LR images!
b A !
downsampling &!phase shift!
!
b A
![Page 41: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/41.jpg)
Pixel Super-Resolution!
I1
I2
ISR
=
ISRb A
!
•! example for 1D scanline!
![Page 42: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/42.jpg)
Pixel Super-Resolution!
• in general: system is well-conditioned for non-integer pixel shifts and super-resolution factors of 2-3x (don’t necessarily need priors)!
• HW 4, Q3: solve (large-scale) pixel super-resolution with gradient
descent to!minimize
ISR
12AISR − b 2
2
![Page 43: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/43.jpg)
HW4 – Q3!
•! gradient descent:!•! use matrix-free functions to implement matrix-vector multiplications!!
!
x = x !"AT Ax ! b( ) = x !"ATrAxAxA !( "ATr
Ax() is already implemented, generate your
own 4 low-res images, then
implement Atx() and solve!
ISR
I1
ISR
I2
SR
ISR
I4
SRISR
I3
SR
![Page 44: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/44.jpg)
Overview of Other Techniques!
![Page 45: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/45.jpg)
Focal Stack!focal stack! contributions!
find highest gradient!
focal stack
find highest gradient
contributions
find highest gradient
all-in-focus image!
•! implemented in a range of products… !
wik
iped
ia!
![Page 46: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/46.jpg)
Aperture Stack!• what changes? exposure and depth of field – extract HDR & depth!!
[Has
inof
f and
Kut
ulak
os 2
007]!
f/2!
f/4!
f/8! refocus
front !
refocus rear!
layered!depth map!
![Page 47: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/47.jpg)
Confocal Stereo!
•! idea: intensity of in-focus point remains constant for varying aperture!
[Has
inof
f and
Kut
ulak
os, 2
006]!
![Page 48: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/48.jpg)
Confocal Stereo!
•! capture aperture and focal stack!
•! for each pixel: find
focus setting where
aperture stack is most invariant!
aperture !"
focus f "
( aperture !i , focus fj )!
[Has
inof
f and
Kut
ulak
os, 2
006]!
![Page 49: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/49.jpg)
Confocal Stereo!aperture !"
focus f "
[Has
inof
f and
Kut
ulak
os, 2
006]!
photograph!estimated depth map!
![Page 50: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/50.jpg)
Low-res High-res Image Pair – Motion Deblurring!
Deblurred image!Blurred image!
Tripod image (Ground Truth) !
slow, high-res camera!
fast, low-res camera!
•! secondary, fast, noisy, low-res camera for motion PSF! estimation!
estimated motion blur!
[Ben
-Ezr
a an
d N
ayar
, 200
3]!
![Page 51: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/51.jpg)
Blurry / Noisy Image Pair – Motion Deblurring!
• same idea, but take two images with same camera!• super short, high ISO noisy exposure for motion PSF estimation!
• longer exposure with camera shake à deblur!
[Yua
n et
al.,
200
7]!
![Page 52: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/52.jpg)
Blurry / Noisy Image Pair – Motion Deblurring!
[Yua
n et
al.,
200
7]!
iteratively motion PSFs!
![Page 53: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/53.jpg)
Flash / No-flash Image Pair!
with flash: not noisy! without flash: noisy, but nice colors!
combined!
[Pet
tsch
nigg
et a
l., 2
004]!
![Page 54: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/54.jpg)
Flash / No-flash Image Pair!no flash!
extract details !(e.g. bilateral filter)!
[Pet
tsch
nigg
et a
l., 2
004]!
Flash / No-flash Image Pairno flash Flash / No-flash Image Pair
flash!
denoised w/!bilateral filter!
![Page 55: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/55.jpg)
Multi-flash Photography!
[Ras
kar e
t al.,
200
4]!
![Page 56: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/56.jpg)
Multi-flash Photography!
[Ras
kar e
t al.,
200
4]!
![Page 57: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/57.jpg)
Multi-flash Photography!
[Ras
kar e
t al.,
200
4]!
![Page 58: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/58.jpg)
Multi-flash Photography!
?!
[Ras
kar e
t al.,
200
4]!
![Page 59: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/59.jpg)
Multi-flash Photography!Multi-flash PhotographyMulti-flash Photography
Canny Intensity !Edge Detection!
Multi-Flash!
Photo! Multi-Flash !Overlay!
[Ras
kar e
t al.,
200
4]!
![Page 60: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/60.jpg)
Multi-flash Photography!
[Ras
kar e
t al.,
200
4]!
![Page 61: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/61.jpg)
Multi-flash Photography!
[Ras
kar e
t al.,
200
4]!
![Page 62: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/62.jpg)
Multi-flash Photography!
[Ras
kar e
t al.,
200
4]!
![Page 63: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/63.jpg)
Multi-flash Photography!Multi-flash Photography
[Ras
kar e
t al.,
200
4]!
![Page 64: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/64.jpg)
Multi-flash Photography!
[Ras
kar e
t al.,
200
4]!
![Page 65: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/65.jpg)
Multi-flash Photography!
[Ras
kar e
t al.,
200
4]!
![Page 66: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/66.jpg)
Multi-flash Photography!
Multi-Flash!Canny!
[Ras
kar e
t al.,
200
4]!
![Page 67: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/67.jpg)
Computational Photography - Overview!
Deb
evec
& M
alik
, 199
7!• high dynamic range!• super-resolution!• focal stack!• aperture stack!• confocal stereo!
• blurry/noisy!• flash/no flash!• multi-flash!à capture and fuse multiple images!
!
![Page 68: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/68.jpg)
Next: Light Field Photography!
•! integral imaging!•! plenoptic 1.0 v 2.0!
•! acquisition!
•! sequential!
•! multiplexing!
•! camera array!•! refocus!
•! Fourier slice theorem!
![Page 69: Burst Photography - Stanford University · Computational Photography - Overview! Debevec & Malik, 1997 • high dynamic range!! • super-resolution! • burst photography! • focal](https://reader033.vdocument.in/reader033/viewer/2022050522/5fa61243684941498d4d4efd/html5/thumbnails/69.jpg)
References and Further Reading!HDR!• Mann, Picard “On Being ‘Undigital’ with Digital Cameras: Extending Dynamic Range by Combining Differently Exposed Pictures”, IS&T 1995 !
• Debevec, Malik, “Recovering High Dynamic Range Radiance Maps from Photographs”, SIGGRAPH 1997!• Robertson, Borman, Stevenson, “Estimation-Theoretic approach to Dynamic Range Improvement Using Multiple Exposures”, Journal of Electronic Imaging 2003!
• Mitsunaga, Nayar, “Radiometric self Calibration”, CVPR 1999!
• Reinhard, Ward, Pattanaik, Debevec (2005). High dynamic range imaging: acquisition, display, and image-based lighting. Elsevier/Morgan Kaufmann!• Fattal, Lischinski, Werman, “Gradient Domain High Dynamic Range Compression”, ACM SIGGRAPH 2002!
• Durand, Dorsey, “Fast Bilateral Filtering for the Display of High Dynamic Range Images”, ACM SIGGRAPH 2002!Super-resolution!
• Baker, Kanade, Limits on super-resolution and how to break them“ IEEE Transactions on Pattern Analysis and Machine Intelligence 24(9), 1167–1183 (2002)!
• Ben-Ezra, Lin, Wilburn, Zhang,, “Penrose pixels for super-resolution” EEE Transactions on Pattern Analysis and Machine Intelligence 33(7), 1370–1383 (2011)!• Ben-Ezra, Zomet, Nayar, “Jitter Camera: High Resolution Video from a Low Resolution Detector”, CVPR 2004!
• Ben-Ezra, Zomet, Nayar, “Video super-resolution using controlled subpixel detector shifts” IEEE Trans. PAMI27(6), 977–987 (2005)!• Elad, Feuer, “Restoration of single super-resolution image from several blurred, noisy and down-sampled measured images” IEEE Trans. Im. Proc. 6(12), (1997)!
Other!
• Ben-Ezra and Nayar, "Motion Deblurring using Hybrid Imaging”, CVPR 2003!• Yuan, Sun, Quan, Shum, “Image Deblurring with Blurred/Noisy Image Pairs”, ACM SIGGRAPH 2007!
• Hasinoff, Kutulakos, “Confocal Stereo”, ECCV 2006!• Hasinoff, Kutulakos, “A Layer-Based Restoration Framework for Variable-Aperture Photography”, ICCV 2007 !
• Raskar, Tan, Feris, Yu, Turk, “Non-photorealistic Camera: Depth Edge Detection and Stylized Rendering using Multi-Flash Imaging”, ACM SIGGRAPH 2004!
• Pettschnigg, Agrawala, Hoppe, Szeliski, Cohen, Toyama, “Digital Photography with Flash and No-Flash Image Pairs”, ACM SIGGRAPH 2004!• Eisemann, Durand, “Flash Photography Enhancement via Intrinsic Relighting”, ACM SIGGRAPH 2004!
!