eecs 442 computer vision cameraswithout cameras we wouldn’t have c.v. • pinhole cameras •...
TRANSCRIPT
EECS 442 – Computer vision
Cameras without cameras we wouldn’t have C.V.
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Reading: [FP] Chapters 1 – 3
[HZ] Chapter 6
Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li
EECS 442 – Computer vision
Cameras without cameras we wouldn’t have C.V.
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Reading: [FP] Chapters 1 – 3
[HZ] Chapter 6
Some slides in this lecture are courtesy to Profs. J. Ponce, S. Seitz, F-F Li
How do we see the world?
• Let’s design a camera – Idea 1: put a piece of film in front of an object
– Do we get a reasonable image?
Pinhole camera
• Add a barrier to block off most of the rays
– This reduces blurring
– The opening known as the aperture
Pinhole perspective projection Pinhole camera f
f = focal length
c = center of the camera
c
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
Some history…
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
Some history…
Photography (Niepce, ―La
Table Servie,‖ 1822)
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
Some history…
Photography (Niepce, ―La
Table Servie,‖ 1822)
Milestones:
• Leonardo da Vinci (1452-1519):
first record of camera obscura
• Johann Zahn (1685): first
portable camera
• Joseph Nicephore Niepce (1822):
first photo - birth of photography
• Daguerréotypes (1839)
• Photographic Film (Eastman, 1889)
• Cinema (Lumière Brothers, 1895)
• Color Photography (Lumière Brothers,
1908)
Some history…
Let’s also not forget…
Motzu
(468-376 BC)
Aristotle
(384-322 BC)
Also: Plato, Euclid
Al-Kindi (c. 801–873)
Ibn al-Haitham
(965-1040) Oldest existent
book on geometry
in China
z
yf'y
z
xf'x
y
xP
z
y
x
P
Pinhole camera
Derived using similar triangles
O
P = [x, z]
P’=[x’, f]
f
z
x
f
x
x
z
Pinhole camera
Common to draw image plane in front of
the focal point. Moving the image plane
merely scales the image.
f f
Pinhole camera
z
yf'y
z
xf'x
Kate lazuka ©
Pinhole camera
Is the size of the aperture important?
Shrinking
aperture
size
- Rays are mixed up
Adding lenses!
-Why the aperture cannot be too small?
-Less light passes through
-Diffraction effect
Cameras & Lenses
• A lens focuses light onto the film
• A lens focuses light onto the film
– Rays passing through the center are not deviated
– All parallel rays converge to one point on a plane
located at the focal length f
focal point
f
Cameras & Lenses
• A lens focuses light onto the film
– There is a specific distance at which objects are ―in
focus‖
[other points project to a ―circle of confusion‖ in the image]
―circle of
confusion‖
Cameras & Lenses
Snell’s law
n1 sin 1 = n2 sin 2
Cameras & Lenses
1 = incident angle
2 = refraction angle
ni = index of refraction
Thin Lenses
Snell’s law:
n1 sin 1 = n2 sin 2
Small angles:
n1 1 n2 2
n1 = n (lens)
n1 = 1 (air)
zo
z
y'z'y
z
x'z'x
ozf'z
)1n(2
Rf
A compound lens Lenses are combined in various ways…
Source wikipedia
f
f
f
Sourc
e w
ikip
edia
Dolly zooms
Compressed perception of depth
Exaggerated perception of depth
Issues with lenses: Chromatic Aberration
• Lens has different refractive indices for different
wavelengths: causes color fringing
)1n(2
Rf
Issues with lenses: Spherical aberration
• Rays farther from the
optical axis focus closer
No distortion
Pin cushion
Barrel (fisheye lens)
Issues with lenses: Radial Distortion
– Deviations are most noticeable for rays that pass through
the edge of the lens
Image magnification
decreases with distance
from the optical axis
Cameras
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Pinhole perspective projection Pinhole camera f
f = focal length
c = center of the camera
c
)z
yf,
z
xf()z,y,x(
2E
3
Is this a linear transformation?
No — division by z is nonlinear
How to make it linear?
)z
yf,
z
xf()z,y,x(
Homogeneous coordinates
homogeneous image
coordinates
homogeneous scene
coordinates
• Converting from homogeneous
coordinates
Perspective Projection
Transformation
1
z
y
x
0100
00f0
000f
z
yf
xf
X
z
yf
z
xf
iX
XMX
M
3H
4
From retina plane to images
Pixels, bottom-left coordinate systems
Coordinate systems
xc
yc
Converting to pixels
x
y
xc
yc
C=[cx, cy]
)cz
yf,c
z
xf()z,y,x( yx
1. Off set
Converting to pixels
)cz
ylf,c
z
xkf()z,y,x( yx
1. Off set
2. From metric to pixels
x
y
xc
yc
C=[cx, cy] Units: k,l : pixel/m
f : m : pixel
, Non-square pixels
Matrix form?
x
y
xc
yc
C=[cx, cy]
)cz
y,c
z
x()z,y,x( yx
Converting to pixels
Camera Matrix
)cz
y,c
z
x()z,y,x( yx
1
z
y
x
0100
0c0
0c0
z
zcy
zcx
X y
x
y
x
x
y
xc
yc
C=[cx, cy]
XMX
X0IK
Camera Matrix
1
z
y
x
0100
0c0
0c0
X y
x
Camera
matrix K
Finite projective cameras
1
z
y
x
0100
0c0
0cs
X y
x
Skew parameter
x
y
xc
yc
C=[cx, cy] K has 5 degrees of freedom!
World reference system
Ow
iw
kw
jw
R,T
•The mapping so far is defined within the camera
reference system
• What if an object is represented in the world
reference system
3D Rotation of Points Rotation around the coordinate axes, counter-clockwise:
100
0cossin
0sincos
)(
cos0sin
010
sin0cos
)(
cossin0
sincos0
001
)(
z
y
x
R
R
R
p
x
Y’
p’
x’
y
z
wXMX wXTRK
World reference system
Ow
iw
kw
jw
R,T
wXTR
X44
10
In 4D homogeneous coordinates:
Internal parameters External parameters
wx XMX 4313
14w4333 XTRK
Projective cameras
Ow
iw
kw
jw
R,T
How many degrees of freedom?
5 + 3 + 3 =11!
100
c0
cs
K y
x
Projective cameras
Ow
iw
kw
jw
R,T
)X
X,
X
X()z,y,x(
w3
w2
w3
w1w
m
m
m
m M is defined up to scale!
Multiplying M by a scalar
won’t change the image
w13 XMX
14w4333 XTRK
3
2
1
m
m
m
M
Theorem (Faugeras, 1993)
][ bATKRKTRKM
3
2
1
a
a
a
A
100
0 y
x
c
cs
K
lf
;kf
Properties of Projection •Points project to points
•Lines project to lines
Properties of Projection
•Distant objects look smaller
z
y'f'y
z
x'f'x
Image plane
scene
Properties of Projection
•Angles are not preserved
•Parallel lines meet! Vanishing point
Vanishing points
• Each set of parallel lines meets at a different point
[The vanishing point for this direction]
•Sets of parallel lines on the same plane lead to collinear
vanishing points [The line is called the horizon for that plane]
One-point perspective • Masaccio, Trinity,
Santa Maria
Novella, Florence,
1425-28
Credit slide S. Lazebnik
Cameras
• Pinhole cameras
• Cameras & lenses
• The geometry of pinhole cameras
• Other camera models
Weak perspective projection
yz
fy
xz
fx
yz
fy
xz
fx
0
0
''
''
''
''
Magnification m
P ~
When the relative scene depth is small compared to its distance from the camera
Orthographic (affine) projection
Distance from center of projection to image plane is infinite
yy
xx
yz
fy
xz
fx
'
'
''
''
Pros and Cons of These Models
• Weak perspective much simpler math. – Accurate when object is small and distant.
– Most useful for recognition.
• Pinhole perspective much more accurate for scenes. – Used in structure from motion.
Qingming Festival by the Riverside Zhang Zeduan ~900 AD
Weak perspective projection
The Kangxi Emperor's Southern Inspection Tour (1691-1698) By Wang Hui
58
http://www.eecs.umich.edu/~silvio/teaching/EECS442_2009/UCSD_videos/3DVisi
on.avi
HW 0.2: watch:
Marilia Maschion Producer & Director UCSD ICAM
undergraduate
Vincent Rabaud Technical Advisor UCSD CSE student
Serge Belongie Project Advisor UCSD CSE Department
Next lecture
•How to calibrate a camera?