photomosaic image stitching using sift...

Photomosaic Image Stitching Using SIFT Features

Liz Del Cid

EGGN 512 Computer Vision

Colorado School of Mines

Image Mosaics (stitching) Objective:

• blend together several overlapping images into one

mosaic (composite)

Inspiration:

• Multiple images can give us depth and 3D structure

o Stereo Vision

o Shape carving

o Structure from motion

Image Stitching Algorithm Overview

For each pair of images:

1. Extract SIFT features

2. Match Features

3. Estimate homography

4. Transform 2nd image

5. Blend two images

6. Combine images

Blending Images Problem:

Seams are produced when adjacent pixel intensities differ in stitched images due to changed lighting conditions or the

vignette effects. (a reduction of an image’s brightness or

saturation at the periphery compared to the image center)

Solution:

Image pyramids

Pyramid Blending

Laplacian pyramid is an algorithm

using Gaussians to blend the image

while keeping the significant features. It

resizes the image into different levels

(sizes) with Gaussian. Then expands the

Gaussians in to the next level and subtracts from the image in that level

to acquire the Laplacian image.

• After creating the Laplacian pyramids for each image, combine the two images in different Laplacian levels by combining partial images from each of them

Pyramid Blending (continued)

Pyramid Blending (continued)

Chaman Singh Verma and Mon-Ju. “Panoramic Image Mosaic”. University of Wisconsin, Madison. Computer Sciences. http://pages.cs.wisc.edu/~csverma/CS766_09/ImageMosaic/imagemosaic.html William Hoff. “Multi-resolution Image Processing”. Colorado School of Mines http://egdegrees.mines.edu/course/eggn510/lectures/19-Multiresolution.pdf

Pyramid Blending Test

Image splicing: no blending

Image Splicing: Laplacian Pyramid

Combining Images After transforming the second image

• Canvas Creation and Placement: o The images are placed on identical sized canvases and then combined

to create the final image.

[transimg xdata ydata] = imtransform(img2, TFORM);

o xdata and ydata are both vectors of two integers. The first integer of each

array is the offset of the initial point of the transformed image. This

information along with the image sizes allowed for not only the summation

of the transformed and static images, but also for an adequately sized

canvas for the resultant image.

• Michael Carroll and Andrew Davidson. “Image Stitching with MatLab”. Louisiana State University.

Department of Electrical and Computer Engineering

Combining Images (continued) • For proper image alignment, chose one of four

different cases to construct a dynamic canvas. The

four cases are:

Combining Images (continued) • Final step:

-imadd()/imsubtract()

• These allow for the combining of the transformed

and static image to create the output image

canvas.

• imsubtract() is first used to clear out the

overlapping area of the final image.

• imadd() then appends the difference between the

images to the area that was overlapping while

keeping the result from imsubtract().

My Results

16 images

≈22 images

7 images

8 images

Recognizing Panoramas

Recognizing Panoramas

System components

• Feature detection and description

• Fast matching (hash table)

• RANSAC filtering of matches

• Intensity-based verification

• Incremental bundle adjustment

M. Brown, R. Szeliski, and S. Winder. Multi-image matching using multi-scale oriented patches, CVPR'2005

Matching Mistakes

• Accidental alignment o repeated / similar regions

• Failed alignments o moving objects / parallax

o low overlap

o “feature-less” regions

More Panoramas

Bibliography • Chaman Singh Verma and Mon-Ju. “Panoramic Image Mosaic”. University of Wisconsin, Madison.

Computer Sciences. http://pages.cs.wisc.edu/~csverma/CS766_09/ImageMosaic/imagemosaic.html

• William Hoff. “Multi-resolution Image Processing”. Colorado School of Mines http://egdegrees.mines.edu/course/eggn510/lectures/19-Multiresolution.pdf

• Michael Carroll and Andrew Davidson. “Image Stitching with MatLab”. Louisiana State University. Department of Electrical and Computer Engineering

• Matthew Brown, Richard Szeliski, and Simon Winder. “Multi-image matching using multi-scale oriented patches”. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'2005), volume I, pages 510-517, June 2005.

• Richard Szeliski. Computer Vision: Algorithms and Applications. Springer, New York, 2010.

• Matthew Brown and David Lowe. “Automatic Panoramic Image Stitching Using Invariant Features”. International Journal of Computer Vision, Vol, No. 3, January 2006 2

Questions?