![Page 1: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/1.jpg)
1
Geometric Transformations and Image Warping:
MosaicingCS 6640
Ross Whitaker, Guido GerigSCI Institute, School of Computing
University of Utah(with slides from: Jinxiang Chai, TAMU)
faculty.cs.tamu.edu/jchai/cpsc641_spring10/lectures/lecture8.ppt
![Page 2: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/2.jpg)
Applications
3
![Page 3: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/3.jpg)
Microscopy (Morane Eye Inst, UofU, T. Tasdizen et al.)
4
![Page 4: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/4.jpg)
5
![Page 5: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/5.jpg)
6
Special Cases
• Nothing new in the scene is uncovered in one view vs another– No ray from the camera gets behind another
1) Pure rotations–arbitrary scene 2) Arbitrary views of planar surfaces
![Page 6: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/6.jpg)
7
Image Homologies
• Images taken under cases 1,2 are perspectively equivalent to within a linear transformation– Projective relationships – equivalence is
![Page 7: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/7.jpg)
Mosaic ProcedureBasic Procedure
– Take a sequence of images from the same position
• Rotate the camera about its optical center
– Compute transformation between second image and first
– Transform the second image to overlap with the first
– Blend the two together to create a mosaic– If there are more images, repeat
8
![Page 8: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/8.jpg)
Image Mosaic
realcamera
syntheticcamera
Can generate any synthetic camera viewas long as it has the same center of projection!
Is a pencil of rays contains all views
9
![Page 9: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/9.jpg)
Image Re-projection
mosaic PPThe mosaic has a natural interpretation in 3D
– The images are reprojected onto a common plane– The mosaic is formed on this plane– Mosaic is a synthetic wide-angle camera 10
![Page 10: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/10.jpg)
Issues in Image Mosaic
mosaic PP
How to relate two images from the same camera center?- image registration
How to re-project images to a common plane?- image warping
?
?
11
![Page 11: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/11.jpg)
Perspective projection equations• 3d world mapped to 2d projection in image plane
Forsyth and Ponce
Camera frame
Image plane
Optical axis
Focal length
Scene / world points
Scene point Image coordinates
‘’‘’
![Page 12: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/12.jpg)
13
3D Perspective and Projection
• Camera model
z
xy
f
Image coordinates
![Page 13: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/13.jpg)
Homogeneous coordinatesIs this a linear transformation?
Trick: add one more coordinate:
homogeneous image coordinates
homogeneous scene coordinates
Converting from homogeneous coordinates
• no—division by z is nonlinear
Slide by Steve Seitz
![Page 14: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/14.jpg)
Perspective Projection Matrix
divide by the third coordinate to convert back to non‐homogeneous coordinates
• Projection is a matrix multiplication using homogeneous coordinates:
'/1
0'/10000100001
fzyx
zyx
f)','(
zyf
zxf
Slide by Steve Seitz
Complete mapping from world points to image pixel positions?
![Page 15: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/15.jpg)
Perspective projection
Camera frame
Extrinsic:Camera frame World frame
World frame
World to camera coord. trans. matrix
(4x4)
Perspectiveprojection matrix
(3x4)
Camera to pixel coord. trans. matrix
(3x3)=
2Dpoint(3x1)
3Dpoint(4x1)
K. Grauman
![Page 16: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/16.jpg)
Rigid Transformations as Mappings
![Page 17: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/17.jpg)
Extrinsic parameters: translation and rotation of camera frame
tpRp CW
WCW
C Non‐homogeneous
coordinates
Homogeneous coordinates
ptRp WC
WC
WC
1000|
|
W. Freeman
Remember discussion of transformations: Rotation and Translation can be Combined into a matrix transformation via homogeneous coordinates!
![Page 18: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/18.jpg)
Transformations
19
affine
x’, y’: homographies
(Geom.) A relation between two figures, such that to any point of the one corresponds one and but one point in the other, and vise versa.
![Page 19: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/19.jpg)
Materials
• Excellent material to derive homography matrix: – www.cs.toronto.edu/~jepson/csc2503/tutor
ial2.pdf– www.cs.toronto.edu/pub/jepson/teaching/vi
sion/2503/tutorial2.pdf
20
![Page 20: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/20.jpg)
21
Perspective Projection Properties
• Lines to lines (linear)
• Conic sections to conic sections
• Convex shapes to convex shapes
• Foreshortening
![Page 21: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/21.jpg)
22
Transforming Images To Make Mosaics
Linear transformation with matrix P
Perspective equivalence Multiply by denominator and reorganize terms
Linear system, solve for P
![Page 22: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/22.jpg)
Transforming Images To Make Mosaics
• Choose sets of corresponding landmarks in two images A and B: xi and x’i
• Calculate matrix P• Transform image A to image B
23
Linear system, solve for P
![Page 23: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/23.jpg)
Image Stitching
Stitch pairs together, blend, then crop
24
![Page 24: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/24.jpg)
Image Stitching
A big image stitched from 5 small images
25
![Page 25: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/25.jpg)
26
Image Mosaicing
![Page 26: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/26.jpg)
27
4 Correspondences
![Page 27: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/27.jpg)
28
5 Correspondences
![Page 28: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/28.jpg)
29
6 Correspondences
![Page 29: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/29.jpg)
30
Mosaicing Issues
• Need a canvas (adjust coordinates/origin)
• Blending at edges of images (avoid sharp transitions)
• Adjusting brightnesses• Cascading transformations
![Page 30: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/30.jpg)
Recognizing panoramas
• A fully automatic 2D image stitcher system
31
![Page 31: Geometric Transformations and Image Warping: Mosaicinggerig/CS6640-F2012/Materials/mosaicing-GG.pdf · 1 Geometric Transformations and Image Warping: Mosaicing CS 6640 Ross Whitaker,](https://reader033.vdocument.in/reader033/viewer/2022052319/5b84a5d17f8b9a317e8c3f44/html5/thumbnails/31.jpg)
Recognizing panoramas
• A fully automatic 2D image stitcher system
- Image matching with SIFT features
- For every image, find the M best images with RANSAC
- Form a graph and find connected component in the graph
- Stitching and blending. 32