lecture 2 camera models - silvio savarese€¦ · silvio savarese lecture 2 - 14-jan-14...
TRANSCRIPT
Lecture 2 -Silvio Savarese 14-Jan-14
Professor Silvio Savarese
Computational Vision and Geometry Lab
Lecture 2Camera Models
Lecture 2 -Silvio Savarese 14-Jan-14
Prerequisites: any questions?
This course requires knowledge of linear algebra, probability, statistics, machine learning and computer vision, as well as decent programming skills. Though not an absolute requirement, it is encouraged and preferred that you have at least taken either CS221 or CS229 or CS131A or have equivalent knowledge.
Topics such as linear filters, feature detectors and descriptors, low level segmentation, tracking, optical flow, clustering and PCA/LDA techniques for recognition won’t be covered in CS231
We will provide links to background material related to CS131A (or discuss during TA sessions) so students can refresh or study those topics if needed
We will leverage concepts from machine learning (CS229) (e.g., SVM, basic Bayesian inference, clustering, etc…) which we won’t cover in this class either. Again, we will supply links to related material for background reading.
Announcements
Lecture 2 -Silvio Savarese 14-Jan-14
Next TA session: Fridays from 2:15-3:05pm
Announcements
Lecture 2 -Silvio Savarese 14-Jan-14
• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras• Other camera models
Lecture 2Camera Models
Reading: [FP] Chapter 1 “Cameras” [FP] Chapter 2 “Geometric Camera Models”[HZ] Chapter 6 “Camera Models”
Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li
How do we see the world?
• Let’s design a camera– Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
Pinhole camera
• Add a barrier to block off most of the rays
– This reduces blurring
– The opening known as the aperture
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
Some history…
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
Some history…
Photography (Niepce, “La
Table Servie,” 1822)
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
Some history…
Photography (Niepce, “La
Table Servie,” 1822)
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
• Daguerréotypes (1839)
• Photographic Film (Eastman, 1889)
• Cinema (Lumière Brothers, 1895)
• Color Photography (Lumière Brothers,
1908)
Some history…
Let’s also not forget…
Motzu
(468-376 BC)
Aristotle
(384-322 BC)
Also: Plato, Euclid
Al-Kindi (c. 801–873)
Ibn al-Haitham
(965-1040)Oldest existent
book on geometry
in China
Pinhole perspective projectionPinhole cameraf
f = focal length
o = aperture = pinhole = center of the camera
o
z
yf'y
z
xf'x
y
xP
z
y
x
P
Pinhole camera
Derived using similar triangles
O
P = [x, z]
P’=[x’, f ]
f
z
x
f
x
i
k
Pinhole camera
Common to draw image plane in front of
the focal point. What’s the transformation between these 2 planes?
f f
Pinhole camera
z
yf'y
z
xf'x
Kate lazuka ©
Pinhole camera
Is the size of the aperture important?
Shrinking
aperture
size
- Rays are mixed up
Adding lenses!
-Why the aperture cannot be too small?
-Less light passes through
-Diffraction effect
Cameras & Lenses
• A lens focuses light onto the film
• A lens focuses light onto the film
– There is a specific distance at which objects are “in
focus”
– Related to the concept of depth of field
“circle of
confusion”
Cameras & Lenses
• A lens focuses light onto the film
– There is a specific distance at which objects are “in
focus”
– Related to the concept of depth of field
Cameras & Lenses
• A lens focuses light onto the film
– All parallel rays converge to one point on a plane
located at the focal length f
– Rays passing through the center are not deviated
focal point
f
Cameras & Lenses
Thin Lenses
Snell’s law:
n1 sin 1 = n2 sin 2
Small angles:
n1 1 n2 2
n1 = n (lens)
n1 = 1 (air)
zo
z
y'z'y
z
x'z'x
ozf'z
)1n(2
Rf
Focal length
For details see lecture
on cameras in CS131A
Issues with lenses: Radial Distortion
Pin cushion
Barrel (fisheye lens)
No distortion
Pin cushion
Barrel (fisheye lens)
Issues with lenses: Radial Distortion
– Deviations are most noticeable for rays that pass through
the edge of the lens
Image magnification
decreases with distance
from the optical axis
Lecture 2 -Silvio Savarese 14-Jan-14
• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras
• Intrinsic• Extrinsic
• Other camera models
Lecture 2Camera Models
Pinhole perspective projectionPinhole cameraf
f = focal length
o = center of the camera
o
)z
yf,
z
xf()z,y,x(
2E
3
From retina plane to images
Pixels, bottom-left coordinate systems
Coordinate systems
xc
yc
Converting to pixels
x
y
xc
yc
C=[cx, cy]
)cz
yf,c
z
xf()z,y,x( yx
1. Off set
Converting to pixels
)cz
ylf,c
z
xkf()z,y,x( yx
1. Off set
2. From metric to pixels
x
y
xc
yc
C=[cx, cy] Units: k,l : pixel/m
f : m: pixel
, Non-square pixels
• Matrix form?
A related question:
• Is this a linear transformation?x
y
xc
yc
C=[cx, cy]
)cz
y,c
z
x()z,y,x( yx
Converting to pixels
Is this a linear transformation?
No — division by z is nonlinear
How to make it linear?
)z
yf,
z
xf()z,y,x(
Homogeneous coordinates
homogeneous image
coordinates
homogeneous scene
coordinates
• Converting from homogeneous
coordinates
For details see lecture on
transformations in CS131A
Camera Matrix
)cz
y,c
z
x()z,y,x( yx
1
z
y
x
0100
0c0
0c0
z
zcy
zcx
X y
x
y
x
x
y
xc
yc
C=[cx, cy]
Perspective Projection
Transformation
1
z
y
x
0100
00f0
000f
z
yf
xf
X
z
yf
z
xf
iX
XMX
M
3H
4
XMX
X0IK
Camera Matrix
1
z
y
x
0100
0c0
0c0
X y
x
Camera
matrix K
Finite projective cameras
1
z
y
x
0100
0c0
0cs
X y
x
Skew parameter
x
y
xc
yc
C=[cx, cy]K has 5 degrees of freedom!
Lecture 2 -Silvio Savarese 14-Jan-14
• Pinhole cameras• Cameras & lenses• The geometry of pinhole cameras
• Intrinsic• Extrinsic
• Other camera models
Lecture 2Camera Models
World reference system
Ow
iw
kw
jwR,T
•The mapping so far is defined within the camera
reference system
• What if an object is represented in the world
reference system
2D Translation
P
P'
t
• For details see lecture on
transformations in CS131A
• See also TA session on Friday
2D Translation Equation
P
x
y
tx
ty
P’
t
)ty,tx(' yx tPP
),(
),(
yx tt
yx
t
P
2D Translation using Homogeneous Coordinates
1100
10
01
1
' y
x
t
t
ty
tx
y
x
y
x
P
)1,,(),(
)1,,(),(
yxyx tttt
yxyx
t
P
t P
1
I tP T P
0
P
x
y
tx
ty
P’
t
Scaling
P
P'
Scaling Equation
P
x
y
sx x
P’sy y
1100
00
00
1
' y
x
s
s
ys
xs
y
x
y
x
P
)1,,(),('
)1,,(),(
ysxsysxs
yxyx
yxyx
P
P
S
'
1
S 0P S P
0
)ys,xs(')y,x( yx PP
Scaling & Translating
'
'' '
P S P
P T P
'' ' ( ) P T P T S P T S P A P
P
P''
Scaling & Translating
1 0 0 0
'' 0 1 0 0
0 0 1 0 0 1 1
0
01
0 0 1 1 1 1
x x
y y
x x x x
y y y y
t s x
t s y
s t x x s x t
s t y y s y t
P T S P
S t
0
Α
Rotation
P
P'
Rotation Equations
• Counter-clockwise rotation by an angle 𝜃
y
x
y
x
cossin
sincos
'
'P
x
y’
P’
x’
y
PRP'
' cos sinx x y
' cos siny y x
Degrees of Freedom
Note: R is an orthogonal matrix and satisfies many interesting properties:
R is 2x2 4 elements
' cos sin
' sin cos
x x
y y
1)det(
R
IRRRRTT
Rotation + Scale + Translation
1 0 cos in 0 0 0
' 0 1 sin cos 0 0 0
0 0 1 0 0 1 0 0 1 1
x x
y y
t s s x
t s y
P T R S P
cos in 0 0
sin cos 0 0
0 0 1 0 0 1 1
x x
y y
s t s x
t s y
1 1 11 1
x x
y y
R t S 0 R S t
0 0 0
If sx = sy, this is asimilarity transformation!
' ( ) P T R S P
3D Rotation of PointsRotation around the coordinate axes, counter-clockwise:
100
0cossin
0sincos
)(
cos0sin
010
sin0cos
)(
cossin0
sincos0
001
)(
z
y
x
R
R
R
p
x
Y’
p’
x’
y
z
wXTRK
World reference system
Ow
iw
kw
jwR,T
wXTR
X
4410
In 4D homogeneous coordinates:
Internal parameters External parameters
XIKX 0' wXTR
IK
4410
0
M
X
X’
wx XMX 4313
14w4333 XTRK
Projective cameras
Ow
iw
kw
jwR,T
How many degrees of freedom?
5 + 3 + 3 =11!
100
c0
cs
K y
x
X
X’
Projective cameras
Ow
iw
kw
jwR,T
w13 XMX
14w4333 XTRK
3
2
1
m
m
m
M
W
W
W
W
m
m
m
m
m
m
X
X
X
X
3
2
1
3
2
1
),(3
2
3
1
w
w
w
w
X
X
X
X
m
m
m
m
E
X
X’
Theorem (Faugeras, 1993)
][ bATKRKTRKM
3
2
1
a
a
a
A
100
0 y
x
c
cs
K
lf
;kf
Properties of Projection• Points project to points
• Lines project to lines
• Distant objects look smaller
Properties of Projection•Angles are not preserved
•Parallel lines meet! Vanishing point
One-point perspective• Masaccio, Trinity,
Santa Maria
Novella, Florence,
1425-28
Credit slide S. Lazebnik
Next lecture
•How to calibrate a camera?