new cs664 computer vision 5. image geometry - cornell university · 2008. 2. 7. · 11 2-d rotation...
TRANSCRIPT
CS664 Computer Vision
5. Image Geometry
Dan Huttenlocher
2
First Assignment
Wells paper handout for question 1Question 2 more open ended– Less accurate approximations
• Simple box filtering doesn’t work
– Anisotropic, spatially dependent
3
Image Warping
T
T
f
f g
g
Image filtering: change range of imageg(x) = T f(x)
Image warping: change domain of image
g(x) = f(T(x))
4
Feature Detection in Images
Filtering to provide area of support– Gaussian, bilateral, …
Measures of local image difference– Edges, corners
More sophisticated features are invariant to certain transformations or warps of the image– E.g., as occur when viewing direction changes
5
Parametric (Global) Warping
Examples of parametric warps:
translation rotation
affine projectivecylindrical
6
Parametric (Global) Warping
p’ = T(p) What does it mean that T is global?– Same function for any point p– Described by a few parameters, often matrix
T
p = (x,y) p’ = (x’,y’)
⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡yx
yx
M''
p’ = M*p
7
Scaling
Scaling a coordinate means multiplying each of its components (axes) by a scalarUniform scaling means this scalar is the same for all components:
× 2
8
Non-uniform scaling: different scalars per component:
Scaling
X × 2,Y × 0.5
9
Scaling
Scaling operation:
Or, in matrix form:
byyaxx
==''
⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡yx
ba
yx
00
''
scaling matrix S
What’s inverse of S?
10
2-D Rotation
θ
(x, y)
(x’, y’)
x’ = x cos(θ) - y sin(θ)y’ = x sin(θ) + y cos(θ)
About origin
11
2-D Rotation
This is easy to capture in matrix form:
Even though sin(θ) and cos(θ) are nonlinear functions of θ,– x’ is a linear combination of x and y– y’ is a linear combination of x and y
Inverse transformation, rotation by –θ– For rotation matrices, det(R) = 1 and
( ) ( )( ) ( ) ⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡ −=⎥
⎦
⎤⎢⎣
⎡yx
yx
θθθθ
cossinsincos
''
TRR =−1
R
12
2x2 Transformation Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Identity? (A rotation)
yyxx
==''
⎥⎦⎤
⎢⎣⎡⎥⎦⎤
⎢⎣⎡=⎥⎦
⎤⎢⎣⎡
yx
yx
1001
''
2D Scale around (0,0)?
ysy
xsx
y
x
*'
*'
=
=⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡yx
ss
yx
y
x
00
''
13
2x2 Transformation Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Rotate around (0,0)?
yxyyxx
*cos*sin'*sin*cos'
Θ+Θ=Θ−Θ= ⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡ΘΘΘ−Θ
=⎥⎦
⎤⎢⎣
⎡yx
yx
cossinsincos
''
2D Shear?
yxshyyshxx
y
x
+=+=
*'*' ⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡yx
shsh
yx
y
x
11
''
14
2x2 Transformation Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Mirror about Y axis?
yyxx
=−=
''
⎥⎦⎤
⎢⎣⎡⎥⎦⎤
⎢⎣⎡−=⎥⎦
⎤⎢⎣⎡
yx
yx
1001
''
2D Mirror over (0,0)?
yyxx
−=−=
''
⎥⎦⎤
⎢⎣⎡⎥⎦⎤
⎢⎣⎡
−−=⎥⎦
⎤⎢⎣⎡
yx
yx
1001
''
15
All 2D Linear Transformations
Linear transformations are combinations of …– Scale,– Rotation,– Shear, and– Mirror
Properties of linear transformations:– Origin maps to origin– Lines map to lines– Parallel lines remain parallel– Ratios are preserved– Closed under composition
⎥⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡yx
dcba
yx
''
⎥⎦⎤
⎢⎣⎡⎥⎦⎤
⎢⎣⎡⎥⎦⎤
⎢⎣⎡⎥⎦⎤
⎢⎣⎡=⎥⎦
⎤⎢⎣⎡
yx
lkji
hgfe
dcba
yx
''
16
Linear Transformations as Change of Basis
Any linear transformation is a basis!
j =(0,1)
i =(1,0)
p
p=4i+3j = (4,3) p’=4u+3vpx’=4ux+3vxpy’=4uy+3vy
v =(vx,vy)
u=(ux,uy)
p’
pp ⎥⎦
⎤⎢⎣
⎡=⎥
⎦
⎤⎢⎣
⎡⎥⎦
⎤⎢⎣
⎡=
yy
xx
yy
xx
vuvu
vuvu
34
'
17
2x2 Transformation Matrices
What types of transformations can be represented with a 2x2 matrix?
2D Translation?
y
x
tyytxx
+=+=
''
Only linear 2D transformations can be represented with a 2x2 matrix
NO!
18
Homogeneous Coordinates
How can we represent translation as matrix?
Homogeneous coordinates– Represent coordinates in 2 dimensions with a 3-
vector
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡⎯⎯⎯⎯ →⎯⎥
⎦
⎤⎢⎣
⎡
1
coords shomogeneou yx
yx
y
x
tyytxx
+=+=
''
19
Homogeneous Coordinates
Add a 3rd coordinate to every 2D point– (x, y, w) represents a point at 2D location
(x/w, y/w)– (x, y, 0) represents a point at infinity– (0, 0, 0) is not allowed
1 2
1
2 (2,1,1) or (4,2,2) or (6,3,3)
x
y
20
Homogeneous Coordinates
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=100
1001
y
x
tt
ranslationT
y
x
tyytxx
+=+=
''
How can we represent translation as matrix?
Last column of homogeneous matrix
21
Translation
Example of translation in homogeneous coordinates
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡++
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
111001001
1''
y
x
y
x
tytx
yx
tt
yx
tx = 2ty = 1
22
Homogeneous 2D Transformations
Basic 2D transformations as 3x3 matrices
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡ΘΘΘ−Θ
=⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
11000cossin0sincos
1''
yx
yx
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
11001001
1''
yx
tt
yx
y
x
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
11000101
1''
yx
shsh
yx
y
x
Translate
Rotate Shear
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡=
⎥⎥⎥
⎦
⎤
⎢⎢⎢
⎣
⎡
11000000
1''
yx
ss
yx
y
x
Scale
23
Affine Transformations
Affine transformations are …– Linear transformations, and– Translations
Properties of affine transformations:– Origin does not necessarily map to origin– Lines map to lines– Parallel lines remain parallel– Ratios are preserved– Closed under composition– Models change of basis– Maps any triangle to any triangle (or parallelogram)
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
wyx
fedcba
wyx
100''
24
Projective Transformations
Projective transformations …– Affine transformations, and– Projective warps
Properties of projective transformations:– Origin does not necessarily map to origin– Lines map to lines– Parallel lines do not necessarily remain parallel– Ratios are not preserved– Closed under composition– Models change of basis– Maps any quadrilateral to any quadrilateral
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
wyx
ihgfedcba
wyx
'''
25
Matrix Composition
Transformations can be combined (composed) by matrix multiplication
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎟⎟⎟
⎠
⎞
⎜⎜⎜
⎝
⎛
⎥⎥⎦
⎤
⎢⎢⎣
⎡
⎥⎥⎦
⎤
⎢⎢⎣
⎡ΘΘΘ−Θ
⎥⎥⎦
⎤
⎢⎢⎣
⎡=
⎥⎥⎦
⎤
⎢⎢⎣
⎡
wyx
sysx
tytx
wyx
1000000
1000cossin0sincos
1001001
'''
p’ = T(tx,ty) R(Θ) S(sx,sy) p
26
2D image transformations
• Nested set of groups
27
Affine Invariants
Affine transformations preserve collinearity of points: – If 3 points lie on the same line, their images under affine
transformations also line on the same line and, – The middle point remains between the other two points
Concurrent lines remain concurrent (images of intersecting lines intersect), Ratio of length of line segments of a given line remains constant Ratio of areas of two triangles remains constant, Ellipses remain ellipses and the same is true for parabolas and hyperbolas
28
3D Interpretations
Orthographic projection plus scaling of Euclidean motion in 3D world yields 2D affine transformation– Often called weak perspective imaging model
Set of coplanar points in 3D– Undergo rigid body motion– Scaling – analogous to perspective size with
distance but same scaling for all the points– Projection into image plane (drop coordinate)
29
Orthographic Projection
Also called “parallel projection”: (x, y, z) → (x, y)What’s the projection matrix (homogeneous coords)?
Image World
30
Weak Perspective Imaging
Scaling in addition to parallel projection
Composed with 3D rigid body motion (6 dof)
Equivalent to affine transformation of plane (6 dof) up to reflection – state without proof
31
Projection – Pinhole Camera Model
Projection equations– Compute intersection with PP of ray from (x,y,z) to COP– Derived using similar triangles– Parallel projection where d infinite
32
Perspective Projection
Homogeneous coordinates
Divide by third coordinate
Scaling projection matrix, no effect
33
Perspective Projection
Composed with 3D rigid body motion (6 dof)Focal length, d, and distance z two additional parametersEquivalent to projective transformation of plane (homography)– 3x3 matrix in homogeneous coordinates, 8 dof– Again state without proof
2D affine and projective transformations correspond to images of plane in space under rigid motion, different imaging models
34
Image Warping
Given coordinate transform (x’,y’) =T(x,y) and source image f(x,y)How do we compute transformed image g(x’,y’) = f(T(x,y))?
x x’
T(x,y)
f(x,y) g(x’,y’)
y y’
35
f(x,y) g(x’,y’)
Forward Warping
Send each pixel f(x,y) to its corresponding location
(x’,y’) = T(x,y) in the second image
x x’
T(x,y)
Q: What about when pixel lands “between” two pixels?
y y’
36
f(x,y) g(x’,y’)
Forward Warping
Send each pixel f(x,y) to its corresponding location
(x’,y’) = T(x,y) in the second image
x x’
T(x,y)
Q: What about when pixel lands “between” two pixels?
y y’
A: Distribute color among neighboring pixels (x’,y’)– Known as “splatting”
Points at pixel centers
37
f(x,y) g(x’,y’)xy
Inverse Warping
Get each pixel g(x’,y’) from its corresponding location
(x,y) = T-1(x’,y’) in the first image
x x’
Q: What about when pixel comes from “between” two pixels?
y’T-1(x,y)
38
f(x,y) g(x’,y’)xy
Inverse Warping
Get each pixel g(x’,y’) from its corresponding location
(x,y) = T-1(x’,y’) in the first image
x x’
T-1(x,y)
Q: What about when pixel comes from “between” two pixels?
y’
A: Interpolate color value from neighbors– Nearest neighbor, bilinear, Gaussian, bicubic
39
Bilinear Interpolation (Reminder)
Sampling at f(x,y):
40
Forward vs. Inverse Warping
Q: Which is better?
A: Usually inverse – eliminates holes– However, requires an invertible warp function – not
always possible...
affine projective
41
Reference
R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, 2nd Ed., Cambridge Univ. Press, 2003.