![Page 1: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/1.jpg)
Image warping and stitchingMay 4th, 2017
Yong Jae LeeUC Davis
![Page 2: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/2.jpg)
Last time
• Interactive segmentation• Feature-based alignment
– 2D transformations– Affine fit– RANSAC
2
![Page 3: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/3.jpg)
Alignment problem
• In alignment, we will fit the parameters of some transformation according to a set of matching feature pairs (“correspondences”).
T
xi
xi '
Slide credit: Adapted by Devi Parikh from Kristen Grauman
3
![Page 4: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/4.jpg)
Main questions
T
T
Warping: Given a source image and a transformation, what does the transformed output look like?
Alignment: Given two images, what is the transformation between them?
Slide credit: Kristen Grauman
4
![Page 5: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/5.jpg)
Motivation for feature-based alignment:Recognition
Figures from David LoweSlide credit: Kristen Grauman
5
![Page 6: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/6.jpg)
6
Slide credit: Kristen Grauman
Motivation for feature-based alignment:Medical image registration
![Page 7: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/7.jpg)
Image from http://graphics.cs.cmu.edu/courses/15-463/2010_fall/
Motivation for feature-based alignment:Image mosaics
Slide credit: Kristen Grauman
7
![Page 8: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/8.jpg)
Parametric (global) warpingExamples of parametric warps:
translation rotation aspect
affineperspective
Source: Alyosha Efros8
![Page 9: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/9.jpg)
Parametric (global) warping
Transformation T is a coordinate-changing machine:p’ = T(p)
What does it mean that T is global?• Is the same for any point p• can be described by just a few numbers (parameters)
Let’s represent T as a matrix:p’ = Mp
T
p = (x,y) p’ = (x’,y’)
=
yx
yx
M''
Source: Alyosha Efros9
![Page 10: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/10.jpg)
Homogeneous coordinates
Converting from homogeneous coordinates
homogeneous image coordinates
To convert to homogeneous coordinates:
10
Slide credit: Kristen Grauman
![Page 11: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/11.jpg)
2D Affine Transformations
Affine transformations are combinations of …• Linear transformations, and• Translations
Parallel lines remain parallel
=
wyx
fedcba
wyx
100'''
Slide credit: Kristen Grauman
11
![Page 12: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/12.jpg)
Projective Transformations
Projective transformations:• Affine transformations, and• Projective warps
Parallel lines do not necessarily remain parallel
=
wyx
ihgfedcba
wyx
'''
Slide credit: Kristen Grauman
12
![Page 13: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/13.jpg)
Fitting an affine transformation• Assuming we know the correspondences, how do we
get the transformation?
),( ii yx ′′),( ii yx
+
=
′′
2
1
43
21
tt
yx
mmmm
yx
i
i
i
i
′′
=
i
i
ii
ii
yx
ttmmmm
yxyx
2
1
4
3
2
1
10000100
Slide credit: Kristen Grauman
13
![Page 14: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/14.jpg)
RANSAC: General form• RANSAC loop:
1. Randomly select a seed group of points on which to base transformation estimate (e.g., a group of matches)
2. Compute transformation from seed group
3. Find inliers to this transformation
4. If the number of inliers is sufficiently large, re-compute estimate of transformation on all of the inliers
• Keep the transformation with the largest number of inliers
Slide credit: Kristen Grauman
14
![Page 15: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/15.jpg)
RANSAC example: Translation
Putative matches
Source: Rick Szeliski15
![Page 16: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/16.jpg)
RANSAC example: Translation
Select one match, count inliers
16
![Page 17: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/17.jpg)
RANSAC example: Translation
Select one match, count inliers
17
![Page 18: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/18.jpg)
RANSAC example: Translation
Find “average” translation vector
18
![Page 19: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/19.jpg)
RANSAC pros and cons• Pros
• Simple and general• Applicable to many different problems• Often works well in practice
• Cons• Lots of parameters to tune• Doesn’t work well for low inlier ratios (too many iterations,
or can fail completely)• Can’t always get a good initialization
of the model based on the minimum number of samples
Lana Lazebnik19
![Page 20: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/20.jpg)
Today
• Image mosaics– Fitting a 2D transformation
• Homography– 2D image warping– Computing an image mosaic
20
![Page 21: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/21.jpg)
HP frames commercial
• http://www.youtube.com/watch?v=2RPl5vPEoQk
21
![Page 22: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/22.jpg)
Mosaics
Obtain a wider angle view by combining multiple images.
image from
S. Seitz
. . .
Slide credit: Kristen Grauman
22
![Page 23: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/23.jpg)
Panoramic Photos are old
• Sydney, 1875
Beirut, late 1800’s
Slide credit: James Hays
![Page 24: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/24.jpg)
How to stitch together a panorama (a.k.a. mosaic)?
• Basic Procedure– Take a sequence of images from the same position
• Rotate the camera about its optical center
– Compute transformation between second image and first– Transform the second image to overlap with the first– Blend the two together to create a mosaic– (If there are more images, repeat)
• …but wait, why should this work at all?– What about the 3D geometry of the scene?– Why aren’t we using it?
Source: Steve Seitz24
![Page 25: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/25.jpg)
Pinhole camera• Pinhole camera is a simple model to approximate
imaging process, perspective projection.
Fig from Forsyth and Ponce
If we treat pinhole as a point, only one ray from any given point can enter the camera.
Virtual image
pinhole
Image plane
Slide credit: Kristen Grauman
25
![Page 26: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/26.jpg)
Mosaics: generating synthetic views
realcamera
syntheticcamera
Can generate any synthetic camera viewas long as it has the same center of projection!
Source: Alyosha Efros
26
![Page 27: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/27.jpg)
Mosaics
Obtain a wider angle view by combining multiple images.
image from
S. Seitz
. . .
Slide credit: Kristen Grauman
27
![Page 28: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/28.jpg)
mosaic PP
Image reprojection
The mosaic has a natural interpretation in 3D• The images are reprojected onto a common plane• The mosaic is formed on this plane• Mosaic is a synthetic wide-angle camera
Source: Steve Seitz28
![Page 29: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/29.jpg)
Image reprojectionBasic question
• How to relate two images from the same camera center?– how to map a pixel from PP1 to PP2
PP2
PP1
Answer• Cast a ray through each pixel in PP1• Draw the pixel where that ray intersects PP2
Observation:Rather than thinking of this as a 3D reprojection, think of it as a 2D image warp from one image to another.
Source: Alyosha Efros29
![Page 30: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/30.jpg)
Image reprojection: Homography
A projective transform is a mapping between any two PPs with the same center of projection• rectangle should map to arbitrary quadrilateral • parallel lines aren’t preserved• but must preserve straight lines
called Homography PP2
PP1
=
1yx
*********
wwy'wx'
H pp’
Source: Alyosha Efros30
![Page 31: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/31.jpg)
(0,0,0)
The projective planeWhy do we need homogeneous coordinates?
• represent points at infinity, homographies, perspective projection, multi-view relationships
What is the geometric intuition?• a point in the image is a ray in projective space
(sx,sy,s)
• Each point (x,y) on the plane is represented by a ray (sx,sy,s)– all points on the ray are equivalent: (x, y, 1) ≅(sx, sy, s)
image plane
(x,y,1)-y
x-z
![Page 32: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/32.jpg)
Homography( )11, yx ( )11, yx ′′
To compute the homography given pairs of corresponding points in the images, we need to set up an equation where the parameters of H are the unknowns…
( )22 , yx ′′( )22 , yx
…
…
( )nn yx , ( )nn yx ′′ ,
Slide credit: Kristen Grauman
32
![Page 33: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/33.jpg)
Solving for homographies
Upto a scale factor.Constraint Frobenius norm of H to be 1.
Problem to be solved:
where vector of unknowns h = [h00,h01,h02,h10,h11,h12,h20,h21,h22]T
=
1222120
121110
020100
yx
hhhhhhhhh
wwy'wx'
p’ = Hp
1
min2
2
=
−
h
bAh
33
..ts
![Page 34: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/34.jpg)
Solving for homographies
=
1222120
121110
020100
i
i
i
i
yx
hhhhhhhhh
w'wy'wx
![Page 35: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/35.jpg)
Solving for homographies
A h 0
Defines a least squares problem:
2n × 9 9 2n
• Since h is only defined up to scale, solve for unit vector ĥ• Solution: ĥ = eigenvector of ATA with smallest eigenvalue• Works with 4 or more points
)1.,.( 2 =hei
![Page 36: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/36.jpg)
Homography
=
1yx
*********
wwy'wx'
H pp’
′′
wyw
wxw
,
( )yx ′′= ,
( )yx,
To apply a given homography H• Compute p’ = Hp (regular matrix multiply)• Convert p’ from homogeneous to image
coordinatesSlide credit: Kristen Grauman 36
![Page 37: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/37.jpg)
Today
• RANSAC for robust fitting – Lines, translation
• Image mosaics– Fitting a 2D transformation
• Homography– 2D image warping– Computing an image mosaic
37
![Page 38: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/38.jpg)
Image warping
Given a coordinate transform and a source image f(x,y), how do we compute a transformed image g(x’,y’) = f(T(x,y))?
x x’
T(x,y)
f(x,y) g(x’,y’)
y y’
Slide from Alyosha Efros, CMU
38
![Page 39: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/39.jpg)
f(x,y) g(x’,y’)
Forward warping
Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image
x x’
T(x,y)
Q: what if pixel lands “between” two pixels?
y y’
Slide from Alyosha Efros, CMU
39
![Page 40: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/40.jpg)
f(x,y) g(x’,y’)
Forward warping
Send each pixel f(x,y) to its corresponding location (x’,y’) = T(x,y) in the second image
x x’
T(x,y)
Q: what if pixel lands “between” two pixels?
y y’
A: distribute color among neighboring pixels (x’,y’)– Known as “splatting”
Slide from Alyosha Efros, CMU
40
![Page 41: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/41.jpg)
f(x,y) g(x’,y’)xy
Inverse warping
Get each pixel g(x’,y’) from its corresponding location (x,y) = T-1(x’,y’) in the first image
x x’
Q: what if pixel comes from “between” two pixels?
y’T-1(x,y)
Slide from Alyosha Efros, CMU
41
![Page 42: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/42.jpg)
f(x,y) g(x’,y’)xy
Inverse warping
Get each pixel g(x’,y’) from its corresponding location (x,y) = T-1(x’,y’) in the first image
x x’
T-1(x,y)
Q: what if pixel comes from “between” two pixels?
y’
A: Interpolate color value from neighbors– nearest neighbor, bilinear…
Slide from Alyosha Efros, CMU
>> help interp242
![Page 43: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/43.jpg)
Bilinear interpolationSampling at f(x,y):
Slide from Alyosha Efros, CMU
43
![Page 44: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/44.jpg)
Recap: How to stitch together a panorama (a.k.a. mosaic)?
• Basic Procedure– Take a sequence of images from the same position
• Rotate the camera about its optical center
– Compute transformation (homography) between second image and first using corresponding points.
– Transform the second image to overlap with the first.– Blend the two together to create a mosaic.– (If there are more images, repeat)
Source: Steve Seitz44
![Page 45: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/45.jpg)
Image warping with homographies
image plane in front image plane belowblack areawhere no pixelmaps to
Source: Steve Seitz
45
![Page 46: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/46.jpg)
Image rectification
pp’
Slide credit: Kristen Grauman
46
![Page 47: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/47.jpg)
Analysing patterns and shapes
Automatically rectified floor
The floor (enlarged)
What is the shape of the b/w floor pattern?
Slide from Antonio Criminisi 47
![Page 48: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/48.jpg)
From Martin Kemp The Science of Art(manual reconstruction)
Aut
omat
ic re
ctifi
catio
n
Analysing patterns and shapes
Slide from Antonio Criminisi48
![Page 49: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/49.jpg)
Automatically rectified floor
St. Lucy Altarpiece, D. Veneziano
Analysing patterns and shapes
What is the (complicated)shape of the floor pattern?
Slide from Criminisi49
![Page 50: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/50.jpg)
From Martin Kemp, The Science of Art(manual reconstruction)
Automaticrectification
Analysing patterns and shapes
Slide from Criminisi50
![Page 51: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/51.jpg)
Julian Beever: Manual Homographies
http://users.skynet.be/J.Beever/pave.htm
![Page 52: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/52.jpg)
Changing camera centerDoes it still work? synthetic PP
PP1
PP2
Source: Alyosha Efros52
![Page 53: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/53.jpg)
Recall: same camera center
realcamera
syntheticcamera
Can generate synthetic camera viewas long as it has the same center of projection.
Source: Alyosha Efros
53
![Page 54: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/54.jpg)
…Or: Planar scene (or far away)
PP3 is a projection plane of both centers of projection, so we are OK!
This is how big aerial photographs are made
PP1
PP3
PP2
Source: Alyosha Efros
54
![Page 55: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/55.jpg)
55
![Page 56: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/56.jpg)
RANSAC for estimating homography
• RANSAC loop:• 1. Select four feature pairs (at random)• 2. Compute homography H (exact)• 3. Compute inliers where SSD(pi’, Hpi)< ε• 4. Keep largest set of inliers• 5. Re-compute least-squares H estimate
on all of the inliers
Slide credit: Steve Seitz
56
![Page 57: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/57.jpg)
Robust feature-based alignment
Source: L. Lazebnik
57
![Page 58: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/58.jpg)
Robust feature-based alignment
• Extract features
Source: L. Lazebnik
58
![Page 59: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/59.jpg)
Robust feature-based alignment
• Extract features• Compute putative matches
Source: L. Lazebnik
59
![Page 60: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/60.jpg)
Robust feature-based alignment
• Extract features• Compute putative matches• Loop:
– Hypothesize transformation T (small group of putative matches that are related by T)
Source: L. Lazebnik
60
![Page 61: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/61.jpg)
Robust feature-based alignment
• Extract features• Compute putative matches• Loop:
– Hypothesize transformation T (small group of putative matches that are related by T)
– Verify transformation (search for other matches consistent with T) Source: L. Lazebnik
61
![Page 62: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/62.jpg)
Robust feature-based alignment
• Extract features• Compute putative matches• Loop:
– Hypothesize transformation T (small group of putative matches that are related by T)
– Verify transformation (search for other matches consistent with T) Source: L. Lazebnik
62
![Page 63: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/63.jpg)
Summary: alignment & warping• Write 2d transformations as matrix-vector
multiplication (including translation when we use homogeneous coordinates)
• Fitting transformations: solve for unknown parameters given corresponding points from two views (affine, projective (homography)).
• Perform image warping (forward, inverse)• Mosaics: uses homography and image warping
to merge views taken from same center of projection.
Slide credit: Kristen Grauman
63
![Page 64: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/64.jpg)
Next time: which features should we match?
Slide credit: Kristen Grauman
64
![Page 65: Image warping and stitching - University of California, Davisyjlee/teaching/ecs174-spring... · 2017-05-04 · Parametric (global) warping Transformation T is a coordinate-changing](https://reader033.vdocument.in/reader033/viewer/2022050116/5f4d5fa986f3570c212b18c6/html5/thumbnails/65.jpg)
Questions?
65