Steve SeitzDept. Computer Science & Eng.

University of Washington

3D Photography:Beyond Perspective

Perspective Projection

Humans evolved with perspective eyes• Capture light along rays that converge at a single point

Cameras also evolved with perspective optics

• Optimized for humans, not computers!

A non-perspective image

The Blue Marble, NASA satellite image composite

The Blue Marble, NASA satellite image composite

The Blue Marble, NASA satellite image composite

Cyberware Scanner

Print Gallery, by M.C. Escher, 1956

Panorama from Disney’s 1940 film Pinocchio (from Wood et al., SIGGRAPH 1997)

What’s an Image?

An image is any 2D subset of rays in space• Actually, the light energy flowing along these rays• Any 2D “slice” of the plenoptic function

perspective image general image

Non-perspective Imaging

Issues:• What other types of images are possible?• Which images are useful?• How can we capture these images?

Path Images

Path Images

Path Images

x

y

t

Path Images

x

y

t

Path Images

Pushbroom images• satellite

Bolles et al. [87]• EPI

Tsuji et al. [92]• omni-directional image

Peleg et al. [97]• manifold mosaic

Radamacher & Bishop [99]• MCOP

. . .

x

y

t

Linear Path

input pushbroom images

Video Cube Demo• application by Michael Cohen et al., Microsoft

Circular Path

panorama(“concentric mosaic”)

circular EPI

input image

Circular Path

input cyclographsvideo cube

What are these images good for?• Applications to computer graphics,

computer vision?

Circular Stereo• [Ishiguro, Yamamoto, Tsuji, 92]

• [Peleg and Ben-Ezra, 99]

• [Shum, Kalai, Seitz, 99]

• [Nayar and Karmarkar, 00]

I

y

x x0 +n

x0 -n x

0

Stereo Panorama

Disparity map result

Stereo Panorama

Dark--close, light--far

Circular Stereo

Advantages• 360 degree scene reconstruction• Uniform accuracy, optimal

Pushbroom Stereo

y

x tx0 +n

x0 -n x

0

Stereo Cyclographs

y

x x0 +n

x0 -n x

0

Stereo Cyclograph Reconstruction

Computed from two cyclograph images• Using unmodified stereo matcher [Zitnick & Kanade]

Stereo Path Images

Do these all produce stereo pairs?• two images with horizontal parallax

Yes

Stereo Path Images

How about this path? No• Yes if the camera path is a conic

– line, circle, ellipse, parabola, hyperbola

• Must capture rays lying on doubly-ruled quadrics– [Padja 2001], [Seitz 2001]

Stereo Parabolic Panoramas

perspective image

parabolic panorama

Beyond PerspectiveCameras for humans, not machines!

• Need to rethink cameras• Image should suit the task

Future: cameras will evolve like CPU’s• First: task-specific cameras• Then: programmable cameras

– FPGA programmable camera arrays

Thanks to Jiwon Kim, Michael Cohen