![Page 1: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/1.jpg)
Layered 3D Tomographic Image Synthesis for Attenuation-
based Light Field and High Dynamic Range Displays
Gordon Wetzstein
Wolfgang Heidrich
Douglas Lanman
Ramesh Raskar
UBC MIT Media Lab
![Page 2: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/2.jpg)
![Page 3: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/3.jpg)
3
Overview
Attenuation Layers
with Spacers
Backlight
![Page 4: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/4.jpg)
4
Key Insights
Multi-Layer Display
Computed
Tomography
4D Light Field
![Page 5: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/5.jpg)
5
Glasses-Free 3D Display
Lenslet Arrays Parallax Barriers Layered 3D
Nin
ten
do 3
DS
Alio
sco
py 3
DH
D 4
2”
Ives 1903 Lippmann 1908
sourc
e: w
ikip
edia
![Page 6: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/6.jpg)
6
Gotoda 2010, 2011
Generalizing Parallax Barriers
mask 1
mask 2
mask 1
mask 2
mask 2
mask 3
mask K
mask 1
… Conventional Parallax Barriers
Lanman et al. 2010
Layered 3D
Tem
pora
l M
ultip
lexin
g
Multiple Layers
We are first to analyze problem and build prototypes
![Page 7: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/7.jpg)
7
Computed Tomography (CT)
sourc
e: w
ikip
edia
x-ray source
x-ray sensor
3D Reconstruction
Reconstructed 2D Slices
![Page 8: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/8.jpg)
8
Tomographic Light Field Synthesis
q
2D Light Field
x
x q
Backlight
Attenuation Volume
Virtual Planes Image Formation
L(x,q ) = e- m (r )dr
cò
c
drrxL )(,log q
P)log( L2
20
P)log(argmin
L
Tomographic Synthesis
![Page 9: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/9.jpg)
9
CT vs. Layered 3D
Computed Tomography Layered 3D
reconstruct physical volume
sensor noise
thin stack of optimized layers
no noise
![Page 10: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/10.jpg)
10
Multi-Layer Decomposition
Input 4D Light Field
Optimized Attenuation Layers
1 2
3 4
5 1 2 3 4 5
Photographs of Prototype
viewer moves right
vie
wer m
oves d
ow
n
![Page 11: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/11.jpg)
11
Depth of Field for 3D Displays Integral Imaging Parallax Barriers
Antialiasing + Display Prefilter Antialiasing + Display Prefilter Zwicker et al. 2006
Display Thickness
Distance of Virtual Plane from Middle of Display (cm)
Cu
toff (
cycle
s/c
m)
Maximum Resolution
![Page 12: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/12.jpg)
12
Integral Imaging Parallax Barriers Layered 3D
?
Distance of Virtual Plane from Middle of Display (cm)
Cu
toff (
cycle
s/c
m)
Display Thickness
Maximum Resolution
How Do Layers Increase Depth of Field?
![Page 13: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/13.jpg)
13
h u
a
backlight Layered 3D
do
ξ
Chai et al. 2000; Durand et al. 2005;
Veeraraghavan et al. 2007;
Lanman et al. 2008; Ihrke et al. 2010
Fourier Transform
u
a
Light Field
fu
fa
f0 -f0
Light Field Spectral Support
attenuator
Review of Frequency-Domain Light Field Analysis
?
![Page 14: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/14.jpg)
14
u
a
h u
a
u
a
Layered 3D
Ä
Fourier Transform
Rear Layer Light Field Front Layer Light Field
fu
fa
f0 -f0
Rear Layer Light Field Spectral Support
fu
fa
f0 -f0
Front Layer Light Field Spectral Support
backlight
attenuator
Multi-Layer Frequency-Domain Analysis
![Page 15: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/15.jpg)
15
u
a
h
u hb
backlight Parallax Barrier
Layered 3D
fu
fa
Emitted Light Field Spectral Support
backlight
attenuator
Multi-Layer Frequency-Domain Analysis
![Page 16: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/16.jpg)
16
u
a
h
fξ
do
ξ
fu
fa
fξ
Layered 3D
Two-Layer Depth of Field
Two-Layer Spectral Support
backlight
attenuator
Distance of Virtual Plane from Middle of Display (cm)
Cuto
ff (
cycle
s/c
m)
Depth of Field
Multi-Layer Frequency-Domain Analysis
![Page 17: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/17.jpg)
17
u
a
h
do
ξ
fu fu
fa fa
fξ
fξ
Layered 3D
Two-Layer Spectral Support Three-Layer Spectral Support
backlight
attenuator
fξ
Distance of Virtual Plane from Middle of Display (cm)
Cuto
ff (
cycle
s/c
m)
Depth of Field
Three-Layer Depth of Field
Multi-Layer Frequency-Domain Analysis
![Page 18: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/18.jpg)
18
Multi-Layer Depth of Field
Conventional*
Layered 3D *Includes integral imaging and parallax barriers
Distance of Virtual Plane from Middle of Display (cm)
Cu
toff (
cycle
s/c
m)
![Page 19: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/19.jpg)
19
Optimization: Number of Layers
Two Layers Three Layers Five Layers
![Page 20: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/20.jpg)
20
Optimization: Display Thickness
Number of Layers
PS
NR
in d
B
Average Reconstruction PSNR for All Scenes
![Page 21: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/21.jpg)
21
Application to HDR Display
“Square Root” Layers
![Page 22: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/22.jpg)
22
Application to HDR Display
“Square Root” Layers
![Page 23: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/23.jpg)
23
Application to HDR Display
Optimized Layers
![Page 24: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/24.jpg)
![Page 25: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/25.jpg)
25
Limitations: Field of View
FOV 10º FOV 20º FOV 45º
![Page 26: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/26.jpg)
26
Personal Glasses-Free 3D Display
Challenges for dynamic display:
Real-time computation
Engineering issues, moiré
![Page 27: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/27.jpg)
27
Dynamic Multi-Layer LCDs
Dynamic Light Field Display using Multi-Layered LCDs, to appear in Siggraph Asia 2011 Douglas Lanman, Gordon Wetzstein, Matthew Hirsch, Wolfgang Heidrich, Ramesh Raskar
![Page 28: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/28.jpg)
28
Photo-stéréo-synthesis Louis Lumière 1920
Focal Stack
Layered
Transparencies 3D effect, but not correct!
Computational Displays
Computational Photography
![Page 29: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/29.jpg)
29
www.layered3d.com
Datasets, code & videos on the website!
![Page 30: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/30.jpg)
30
![Page 31: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/31.jpg)
31
3D objects only inside enclosure
mostly mechanically moving parts
all depth cues
computationally expensive Holograms
inexpensive fabrication
computationally efficient
3D objects outside enclosure
no moving parts
Volumetric Displays
Glasses-Free 3D Display
Zebra Imaging MIT Holovideo Jones et al. 2007 Sony LightSpace
Layered 3D
![Page 32: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/32.jpg)
32
Limitations: “Flip Animations”
Parallax Barriers Multi-Layer
Full Resolution
Multi-Layer
Reduced Resolution
![Page 33: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/33.jpg)
33
Performance Assessment
Prototype Prototype Prototype
Input Input Input Expected Error Expected Error Expected Error
![Page 34: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/34.jpg)
34
Application to HDR Display
bottom bottom
bottom bottom
top top
top top
Se
etz
en
20
04
, D
olb
y C
an
ad
a
LCD
LED Array
![Page 35: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/35.jpg)
35
Implementation
Prototype in Front of LCD (Backlight)
Epson Stylus 2200 inkjet (300 dpi, six color primaries)
5 layers (5.7×7.6 cm),1.27 cm thickness, 10° field of view
Transparencies and Acrylic Spacers Inkjet Transparency Printer *
*not „90s, try eBay
![Page 36: Gordon Wetzstein Douglas Lanman Wolfgang Heidrich Ramesh ... · UBC MIT Media Lab. 3 Overview Attenuation Layers with Spacers Backlight . 4 Key Insights Multi-Layer Display Computed](https://reader033.vdocument.in/reader033/viewer/2022050510/5f9b60fec0ec352f47108f1e/html5/thumbnails/36.jpg)
36
Implementation: Software viewer moves right
vie
wer m
oves d
ow
n 1
2 3
4 5
1 2 3
4 5
POV-Ray: 7×7 views (512×384 pixels), 10° field of view Depth of field tuned for combined antialiasing and display prefilter
MATLAB: LSQLIN (independently for each color channel)
12 minutes on 2.4 GHz Intel Core 2 with 8 GB RAM