nonphotorealistic rendering, and organization of npr methods...
TRANSCRIPT
1
Nonphotorealistic rendering, and future cameras
Computational Photography, 6.882
Bill FreemanFredo Durand
May 11, 2006
Organization of NPR methods
• Automated methods– 2-d processing– 3-d processing
• Interactive methods– 2-d processing– 3-d processing
Computer generated watercolor
http://www.cs.utah.edu/npr/papers/npr course Sig99.pdf http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf
http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf
2
http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf
http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf
Interactive application.
But too slow to let you
paint in real-time.
http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf
Offline application
http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf
User inputs
3
http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf
Steps in the rendering
(user-supplied region mask not
shown)
Resulting watercolor
http://citeseer.ist.psu.edu/cache/papers/cs/12745/http:zSzzSzcdserver.icemt.iastate.eduzSzcdzSzs97cpzSzcontentszSzpaperszSzcurtiszSzcurtis.pdf/curtis97computergenerated.pdf
Source image for a 3d animation
4
Pen and ink illustration, exploiting 3d geometry
http
://w
ww
.cs.u
tah.
edu/
npr/p
aper
s/np
r_co
urse
_Sig
99.p
df
http://www.cs.utah.edu/npr/papers/npr_course_Sig99.pdf
The artist approved of this one…
http://www.cs.utah.edu/npr/papers/npr_course_Sig99.pdf
Future cameras
Computational Photography, 6.882
Bill FreemanFredo Durand
May 11, 2006
5
What can be improved about current cameras?
• Dynamic range• Blurred photos• Post-shot controllable depth of field• Post-shot editable lighting, positions, etc.• Size of camera
(your list first…)
What crazy other things?
• The previous list is all mostly with reference to the functionality of a film camera. Surely unexpected camera capabilities and uses, only possible with digital media, will come with future cameras.
Some possible future directions
• Assorted pixels• Foveon imager• Coded shutter flutter• Light field camera• Gradient camera
Some possible future directions
• Assorted pixels• Foveon imager• Coded shutter flutter• Light field camera• Gradient camera
6
Color pixel mosaic
http://www1.cs.columbia.edu/CAVE/publications/pdfs/Narasimhan_PAMI05.pdf
Intensity attenuation mosaic
http://www1.cs.columbia.edu/CAVE/publications/pdfs/Narasimhan_PAMI05.pdf
Color and intensity mosaic
http://www1.cs.columbia.edu/CAVE/publications/pdfs/Narasimhan_PAMI05.pdf
Color and polarization mosaic
http://www1.cs.columbia.edu/CAVE/publications/pdfs/Narasimhan_PAMI05.pdf
Temporal sensitivity modulation
http://www1.cs.columbia.edu/CAVE/publications/pdfs/Narasimhan_PAMI05.pdf Shree Nayar, Columbia University
7
http://www1.cs.columbia.edu/CAVE/publications/pdfs/Narasimhan_PAMI05.pdfhttp://www1.cs.columbia.edu/CAVE/publications/pdfs/Narasimhan_PAMI05.pdf
http://www1.cs.columbia.edu/CAVE/publications/pdfs/Narasimhan_PAMI05.pdf
http
://w
ww
1.cs
.col
umbi
a.ed
u/C
AVE
/pub
licat
ions
s/N
aras
imha
n_P
AM
I05.
http
://w
ww
1.cs
.col
umbi
a.ed
u/C
AVE
/pub
licat
ions
s/N
aras
imha
n_P
AM
I05.
http
://w
ww
1.cs
.col
umbi
a.ed
u/C
AVE
/pub
licat
ions
s/N
aras
imha
n_P
AM
I05.
8
Original (12 bits)
http
://w
ww
1.cs
.col
umbi
a.ed
u/C
AVE
/pub
licat
ions
s/N
aras
imha
n_P
AM
I05.
pdf As sampled (8 bits)
http
://w
ww
1.cs
.col
umbi
a.ed
u/C
AVE
/pub
licat
ions
s/N
aras
imha
n_P
AM
I05.
Cubic spline interpolation to 12 bits
http
://w
ww
1.cs
.col
umbi
a.ed
u/C
AVE
/pub
licat
ions
s/N
aras
imha
n_P
AM
I05.
Linear regression interpolation to 12 bits
http
://w
ww
1.cs
.col
umbi
a.ed
u/C
AVE
/pub
licat
ions
s/N
aras
imha
n_P
AM
I05.
Some possible future directions
• Assorted pixels• Foveon imager• Coded shutter flutter• Light field camera• Gradient camera
http://www.foveon.com/
9
http://www.foveon.com/files/CIC13_Hubel_Final.pdf http://www.foveon.com/files/CIC13_Hubel_Final.pdf
http://www.foveon.com/ http://www.foveon.com/
http://www.foveon.com/ http://www.foveon.com/
10
http://www.foveon.com/
Foveon features• Use the optical properties of silicon itself to
separate colors.– Different wavelengths get absorbed at different
depths of the silicon—blue, then green, then red.• More efficient at capturing light—don’t discard
2/3 of the spectrum at each pixel.• Variable pixel size, depending on photo mode or
video mode.
• 2002: “…destined to become the standard in image sensors for electronic cameras.”, said Carver Mead, Foveon’s founder. Status now…?
http://www.foveon.com/
Some possible future directions
• Assorted pixels• Foveon imager• Coded shutter flutter• Light field camera• Gradient camera
11
Blurring convolution kernelFourier transforms into sinc
Fourier transforms into a function with no zeros
Some possible future directions
• Assorted pixels• Foveon imager• Coded shutter flutter• Light field camera• Gradient camera
http://graphics.stanford.edu/papers/lfcamera/lfcamera-150dpi.pdf
12
http://graphics.stanford.edu/papers/lfcamera/lfcamera-150dpi.pdf
http
://gr
aphi
cs.s
tanf
ord.
edu/
pape
rs/lf
cam
era/
lfcam
era-
150d
pi.p
df
http
://gr
aphi
cs.s
tanf
ord.
edu/
pape
rs/lf
cam
era/
lfcam
era-
150d
pi.p
df
http
://gr
aphi
cs.s
tanf
ord.
edu/
pape
rs/lf
cam
era/
lfcam
era-
150d
pi.p
df
http
://gr
aphi
cs.s
tanf
ord.
edu/
pape
rs/lf
cam
era/
lfcam
era-
150d
pi.p
df
http
://gr
aphi
cs.s
tanf
ord.
edu/
pape
rs/lf
cam
era/
lfcam
era-
150d
pi.p
df
13
http
://gr
aphi
cs.s
tanf
ord.
edu/
pape
rs/lf
cam
era/
lfcam
era-
150d
pi.p
df
http
://gr
aphi
cs.s
tanf
ord.
edu/
pape
rs/lf
cam
era/
lfcam
era-
150d
pi.p
df
Some possible future directions
• Assorted pixels• Foveon imager• Coded shutter flutter• Light field camera• Gradient camera
14
end