cs 691b computational photography instructor: gianfranco doretto 3d to 2d projections
TRANSCRIPT
CS 691B Computational Photography
Instructor: Gianfranco Doretto3D to 2D Projections
Pinhole camera model
• Pinhole model:– Captures pencil of rays – all rays through a single point– The point is called Center of Projection (COP)– The image is formed on the Image Plane– Effective focal length f is distance from COP to Image
Plane
Point of observation
Dimensionality Reduction Machine (3D to 2D)
3D world 2D image
What have we lost?• Angles• Distances (lengths)
Projection can be tricky…
Slide source: Seitz
Projection can be tricky…
Slide source: Seitz
Projective Geometry
What is lost?• Length
Which is closer?
Who is taller?
Lengths can’t be trusted...
B’
C’
A’
…but humans adopt!
http://www.michaelbach.de/ot/sze_muelue/index.html
Müller-Lyer Illusion
We don’t make measurements in the image plane
Projective Geometry
What is lost?• Length• Angles
Perpendicular?
Parallel?
Parallel lines aren’t…
Projective Geometry
What is preserved?• Straight lines are still straight
Vanishing points and lines
Parallel lines in the world intersect in the image at a “vanishing point”
Vanishing points and lines
oVanishing Point oVanishing Point
Vanishing Line
Vanishing points and lines
Vanishing point
Vanishing line
Vanishing point
Vertical vanishing point
(at infinity)
Credit: Criminisi
Vanishing points and lines
Photo from online Tate collection
Note on estimating vanishing points
Use multiple lines for better accuracy… but lines will not intersect at exactly the same point in practice
One solution: take mean of intersecting pairs… bad idea!
Instead, minimize angular differences
Vanishing objects
Modeling projection
• The coordinate system– We will use the pin-hole model as an approximation– Put the optical center (Center Of Projection) at the origin– Put the image plane (Projection Plane) in front of the COP
• Why?
– The camera looks down the negative z axis• we need this if we want right-handed-coordinates
–
Modeling projection
• Projection equations– Compute intersection with PP of ray from (x,y,z) to COP– Derived using similar triangles (on board)
• We get the projection by throwing out the last coordinate:
Homogeneous coordinates• Is this a linear transformation?
Trick: add one more coordinate:
homogeneous image coordinates
homogeneous scene coordinates
Converting from homogeneous coordinates
• no—division by z is nonlinear
Homogeneous coordinates
Invariant to scaling
Point in Cartesian is ray in Homogeneous
wywx
kwkykwkx
kw
ky
kx
w
y
x
k
Homogeneous Coordinates
Cartesian Coordinates
Perspective Projection• Projection is a matrix multiply using homogeneous coordinates:
divide by third coordinate
This is known as perspective projection• The matrix is the projection matrix• Can also formulate as a 4x4
divide by fourth coordinate
Orthographic Projection
• Special case of perspective projection– Distance from the COP to the PP is infinite
– Also called “parallel projection”– What’s the projection matrix?
Image World
Scaled Orthographic Projection
• Special case of perspective projection– Object dimensions are small compared to
distance to camera
– Also called “weak perspective”– What’s the projection matrix?
1000
000
000
z
y
x
s
f
f
w
v
u
Spherical Projection
• What if PP is spherical with center at COP?• In spherical coordinates, projection is trivial:• (θ,Φ) = (θ,Φ,d)• Note: doesn’t depend on focal length!
Projection matrix
XtRKx x: Image Coordinates: w(u,v,1)K: Intrinsic Matrix (3x3)R: Rotation (3x3) t: Translation (3x1)X: World Coordinates: (X,Y,Z,1)
Ow
iw
kw
jw
R,T
X0IKx
10100
000
000
1z
y
x
f
f
v
u
w
K
Projection matrix
Intrinsic Assumptions• Unit aspect ratio• Principal point at (0,0)• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
Remove assumption: known optical center
X0IKx
10100
00
00
10
0
z
y
x
vf
uf
v
u
w
Intrinsic Assumptions• Unit aspect ratio• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
Remove assumption: square pixels
X0IKx
10100
00
00
10
0
z
y
x
v
u
v
u
w
Intrinsic Assumptions• No skew
Extrinsic Assumptions• No rotation• Camera at (0,0,0)
Remove assumption: non-skewed pixels
X0IKx
10100
00
0
10
0
z
y
x
v
us
v
u
w
Intrinsic Assumptions Extrinsic Assumptions• No rotation• Camera at (0,0,0)
Note: different books use different notation for parameters
Oriented and Translated Camera
Ow
iw
kw
jw
t
R
Allow camera translation
XtIKx
1100
010
001
100
0
0
10
0
z
y
x
t
t
t
v
u
v
u
w
z
y
x
Intrinsic Assumptions Extrinsic Assumptions• No rotation
3D Rotation of Points
Rotation around the coordinate axes, counter-clockwise:
100
0cossin
0sincos
)(
cos0sin
010
sin0cos
)(
cossin0
sincos0
001
)(
z
y
x
R
R
R
p
p’
y
z
Slide Credit: Saverese
Allow camera rotation
XtRKx
1100
0
1 333231
232221
131211
0
0
z
y
x
trrr
trrr
trrr
v
us
v
u
w
z
y
x
Degrees of freedom
XtRKx
1100
0
1 333231
232221
131211
0
0
z
y
x
trrr
trrr
trrr
v
us
v
u
w
z
y
x
5 6
Vanishing Point = Projection from Infinity
R
R
R
z
y
x
z
y
x
z
y
x
KpKRptRKp
0
R
R
R
z
y
x
vf
uf
v
u
w
100
0
0
10
0 0uz
fxu
R
R
0vz
fyv
R
R
Lens Flaws
Lens Flaws: Chromatic Aberration
• Dispersion: wavelength-dependent refractive index– (enables prism to spread white light beam into rainbow)
• Modifies ray-bending and lens focal length: f(λ)
• color fringes near edges of image• Corrections: add ‘doublet’
lens of flint glass, etc.
Chromatic Aberration
Near Lens Center Near Lens Outer Edge
Radial Distortion (e.g. ‘Barrel’ and ‘pin-cushion’)
straight lines curve around the image center
Radial Distortion
• Radial distortion of the image– Caused by imperfect lenses– Deviations are most noticeable for rays
that pass through the edge of the lens
No distortion Pin cushion Barrel
Radial Distortion
Slide Credits
• This set of sides also contains contributionskindly made available by the following authors– Alexei Efros– Stephen E. Palmer– Steve Seitz– Derek Hoiem– David Forsyth– George Bebis– Silvio Savarese