image warping cse399b, spring 07 computer vision · 2007-01-30 · parametric (global) warping...
TRANSCRIPT
![Page 1: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/1.jpg)
http://maps.a9.com
Image Warping
CSE399b, Spring 07
Computer Vision
http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html
![Page 2: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/2.jpg)
http://www.cs.ubc.ca/~mbrown/autostitch/autostitch.html
![Page 3: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/3.jpg)
Autostiching on A9.com images,
Spruce street, Philadelphia
![Page 4: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/4.jpg)
![Page 5: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/5.jpg)
![Page 6: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/6.jpg)
![Page 7: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/7.jpg)
![Page 8: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/8.jpg)
![Page 9: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/9.jpg)
Image Warping
Slides from 15-463: Computational Photography
Alexei Efros, CMU, Fall 2005
Some slides from Steve Seitz
![Page 10: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/10.jpg)
Image Warping
image filtering: change range of image
g(x) = T(f(x))
f
x
T
f
x
f
x
Tf
x
image warping: change domain of image
g(x) = f(T(x))
![Page 11: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/11.jpg)
Image Warping
T
T
f
f g
g
image filtering: change range of image
g(x) = T(f(x))
image warping: change domain of image
g(x) = f(T(x))
![Page 12: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/12.jpg)
Parametric (global) warping
Examples of parametric warps:
translatio
n
rotation aspect
affineperspective
cylindrical
![Page 13: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/13.jpg)
Parametric (global) warping
Transformation T is a coordinate-changing machine:
p’ = T(p)
What does it mean that T is global?
• Is the same for any point p
• can be described by just a few numbers (parameters)
Let’s represent T as a matrix:
p’ = M*p
T
p = (x,y) p! = (x!,y!)
!"
#$%
&=!
"
#$%
&
y
x
y
xM
'
'
![Page 14: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/14.jpg)
Scaling
Scaling a coordinate means multiplying each of its components bya scalar
Uniform scaling means this scalar is the same for all components:
! 2
![Page 15: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/15.jpg)
Non-uniform scaling: different scalars per component:
Scaling
X ! 2,
Y ! 0.5
![Page 16: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/16.jpg)
Scaling
Scaling operation:
Or, in matrix form:
byy
axx
=
=
'
'
!"
#$%
&!"
#$%
&=!
"
#$%
&
y
x
b
a
y
x
0
0
'
'
scaling matrix S
What’s inverse of S?
![Page 17: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/17.jpg)
2-D Rotation
"
(x, y)
(x!, y!)
x! = x cos(") - y sin(")
y! = x sin(") + y cos(")
![Page 18: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/18.jpg)
2-D Rotation
x = r cos (#)
y = r sin (#)x! = r cos (# + ")
y! = r sin (# + ")
Trig Identity…x! = r cos(#) cos(") – r sin(#) sin(")y! = r sin(#) sin(") + r cos(#) cos(")
Substitute…x! = x cos(") - y sin(")y! = x sin(") + y cos(")
"
(x, y)
(x!, y!)
#
![Page 19: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/19.jpg)
2-D Rotation
This is easy to capture in matrix form:
Even though sin(") and cos(") are nonlinear functions of ",
• x’ is a linear combination of x and y
• y’ is a linear combination of x and y
What is the inverse transformation?
• Rotation by –"
• For rotation matrices, det(R) = 1 so
( ) ( )
( ) ( ) !"
#$%
&!"
#$%
& '=!
"
#$%
&
y
x
y
x
((
((
cossin
sincos
'
'
TRR =
!1
R
![Page 20: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/20.jpg)
2x2 Matrices
What types of transformations can be
represented with a 2x2 matrix?
2D Identity?
yyxx
==''
!"#
$%&!"#
$%&=
!"#
$%&
yx
yx
1001
''
2D Scale around (0,0)?
ysy
xsx
y
x
*'
*'
=
=
!"
#$%
&!"
#$%
&=!
"
#$%
&
y
x
s
s
y
x
y
x
0
0
'
'
![Page 21: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/21.jpg)
2x2 Matrices
What types of transformations can be
represented with a 2x2 matrix?
2D Rotate around (0,0)?
yxyyxx
*cos*sin'*sin*cos'
!+!=!"!=
!"
#$%
&!"
#$%
&
''
'('=!
"
#$%
&
y
x
y
x
cossin
sincos
'
'
2D Shear?
yxshy
yshxx
y
x
+=
+=
*'
*'
!"
#$%
&!"
#$%
&=!
"
#$%
&
y
x
sh
sh
y
x
y
x
1
1
'
'
![Page 22: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/22.jpg)
2x2 Matrices
What types of transformations can be
represented with a 2x2 matrix?
2D Mirror about Y axis?
yyxx
=!=
''
!"#
$%&!"#
$%&'=
!"#
$%&
yx
yx
1001
''
2D Mirror over (0,0)?
yyxx!=!=
''
!"#
$%&!"#
$%&
''
=!"#
$%&
yx
yx
1001
''
![Page 23: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/23.jpg)
2x2 Matrices
What types of transformations can be
represented with a 2x2 matrix?
2D Translation?
y
x
tyy
txx
+=
+=
'
'
Only linear 2D transformations
can be represented with a 2x2 matrix
NO!
![Page 24: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/24.jpg)
All 2D Linear Transformations
Linear transformations are combinations of …
• Scale,
• Rotation,
• Shear, and
• Mirror
Properties of linear transformations:
• Origin maps to origin
• Lines map to lines
• Parallel lines remain parallel
• Ratios are preserved
• Closed under composition
!"
#$%
&!"
#$%
&=!
"
#$%
&
y
x
dc
ba
y
x
'
'
!"#
$%&!"
#$%
&!"
#$%
&!"#
$%&=
!"#
$%&
yx
lkji
hgfe
dcba
yx''
![Page 25: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/25.jpg)
Linear Transformations as Change of Basis
Any linear transformation is a basis!!!• What’s the inverse transform?
• How can we change from any basis to any basis?
• What if the basis are orthogonal?
j =(0,1)
i =(1,0)
p
p=4i+3j = (4,3) p’=4u+3vpx’=4ux+3vx
py’=4uy+3vy
v =(vx,vy)
u=(ux,uy)
p’
pp !"
#$%
&=!
"
#$%
&!"
#$%
&=
yy
xx
yy
xx
vu
vu
vu
vu
3
4'
![Page 26: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/26.jpg)
Homogeneous Coordinates
Q: How can we represent translation as a 3x3
matrix?
y
x
tyy
txx
+=
+=
'
'
![Page 27: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/27.jpg)
Homogeneous Coordinates
Homogeneous coordinates
• represent coordinates in 2
dimensions with a 3-vector
!!!
"
#
$$$
%
&
''''' ('!"
#$%
&
1
coords shomogeneou y
x
y
x
![Page 28: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/28.jpg)
Homogeneous Coordinates
Q: How can we represent translation as a 3x3
matrix?
A: Using the rightmost column:
!!!
"
#
$$$
%
&
=
100
10
01
y
x
t
t
ranslationT
y
x
tyy
txx
+=
+=
'
'
![Page 29: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/29.jpg)
Translation
Example of translation
!!!
"
#
$$$
%
&
+
+
=
!!!
"
#
$$$
%
&
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
11100
10
01
1
'
'
y
x
y
x
ty
tx
y
x
t
t
y
x
tx = 2
ty = 1
Homogeneous Coordinates
![Page 30: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/30.jpg)
Homogeneous Coordinates
Add a 3rd coordinate to every 2D point
• (x, y, w) represents a point at location (x/w, y/w)
• (x, y, 0) represents a point at infinity
• (0, 0, 0) is not allowed
Convenient
coordinate system to
represent many
useful
transformations
1 2
1
2(2,1,1) or (4,2,2) or (6,3,3)
x
y
![Page 31: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/31.jpg)
Basic 2D Transformations
Basic 2D transformations as 3x3 matrices
!!!
"
#
$$$
%
&
!!!
"
#
$$$
%
&
''
'('
=
!!!
"
#
$$$
%
&
1100
0cossin
0sincos
1
'
'
y
x
y
x
!!!
"
#
$$$
%
&
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
1100
10
01
1
'
'
y
x
t
t
y
x
y
x
!!!
"
#
$$$
%
&
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
1100
01
01
1
'
'
y
x
sh
sh
y
x
y
x
Translate
Rotate Shear
!!!
"
#
$$$
%
&
!!!
"
#
$$$
%
&
=
!!!
"
#
$$$
%
&
1100
00
00
1
'
'
y
x
s
s
y
x
y
x
Scale
![Page 32: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/32.jpg)
Affine Transformations
Affine transformations are combinations of …
• Linear transformations, and
• Translations
Properties of affine transformations:
• Origin does not necessarily map to origin
• Lines map to lines
• Parallel lines remain parallel
• Ratios are preserved
• Closed under composition
• Models change of basis
!!
"
#
$$
%
&
!!
"
#
$$
%
&=
!!
"
#
$$
%
&
wyx
fedcba
wyx
100''
![Page 33: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/33.jpg)
Projective Transformations
Projective transformations …
• Affine transformations, and
• Projective warps
Properties of projective transformations:
• Origin does not necessarily map to origin
• Lines map to lines
• Parallel lines do not necessarily remain parallel
• Ratios are not preserved
• Closed under composition
• Models change of basis
!!
"
#
$$
%
&
!!
"
#
$$
%
&=
!!
"
#
$$
%
&
wyx
ihgfedcba
wyx
'''
![Page 34: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/34.jpg)
Matrix Composition
Transformations can be combined by
matrix multiplication
!!
"
#
$$
%
&
'''
(
)
***
+
,
!!
"
#
$$
%
&
!!"
#
$$%
&---.-
!!
"
#
$$
%
&=
!!
"
#
$$
%
&
wyx
sysx
tytx
wyx
1000000
1000cossin0sincos
1001001
'''
p’ = T(tx,ty) R($) S(sx,sy) p
![Page 35: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/35.jpg)
2D image transformations
These transformations are a nested set of groups
• Closed under composition and inverse is a member
![Page 36: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/36.jpg)
Image warping
Given a coordinate transform (x’,y’) = h(x,y) and a
source image f(x,y), how do we compute a
transformed image g(x’,y’) = f(T(x,y))?
x x!
T(x,y)
f(x,y) g(x!,y!)
y y!
![Page 37: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/37.jpg)
f(x,y) g(x!,y!)
Forward warping
Send each pixel f(x,y) to its corresponding location
(x’,y’) = T(x,y) in the second image
x x!
T(x,y)
Q: what if pixel lands “between” two pixels?
y y!
![Page 38: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/38.jpg)
f(x,y) g(x!,y!)
Forward warping
Send each pixel f(x,y) to its corresponding location
(x’,y’) = T(x,y) in the second image
x x!
T(x,y)
Q: what if pixel lands “between” two pixels?
y y!
A: distribute color among neighboring pixels (x’,y’)
– Known as “splatting”
![Page 39: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/39.jpg)
f(x,y) g(x!,y!)x
y
Inverse warping
Get each pixel g(x’,y’) from its corresponding location
(x,y) = T-1(x’,y’) in the first image
x x!
Q: what if pixel comes from “between” two pixels?
y!T-1(x,y)
![Page 40: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/40.jpg)
f(x,y) g(x!,y!)x
y
Inverse warping
Get each pixel g(x’,y’) from its corresponding location
(x,y) = T-1(x’,y’) in the first image
x x!
T-1(x,y)
Q: what if pixel comes from “between” two pixels?
y!
A: Interpolate color value from neighbors– nearest neighbor, bilinear, Gaussian, bicubic
![Page 41: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/41.jpg)
Bilinear interpolation
Sampling at f(x,y):
![Page 42: Image Warping CSE399b, Spring 07 Computer Vision · 2007-01-30 · Parametric (global) warping Transformation T is a coordinate-changing machine: pÕ = T(p) What does it mean that](https://reader034.vdocument.in/reader034/viewer/2022042803/5f4d5fa986f3570c212b18c7/html5/thumbnails/42.jpg)
Forward vs. inverse warping
Q: which is better?
A: usually inverse—eliminates holes• however, it requires an invertible warp function—not always possible...