computer vision spring 2012 15-385,-685 instructor: s. narasimhan wean 5409 t-r 10:30am – 11:50am
TRANSCRIPT
![Page 1: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/1.jpg)
Computer Vision
Spring 2012 15-385,-685
Instructor: S. Narasimhan
Wean 5409
T-R 10:30am – 11:50am
![Page 2: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/2.jpg)
Image Resampling and Pyramids
Lecture #8
![Page 3: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/3.jpg)
Image Scaling
This image is too big tofit on the screen. Howcan we reduce it?
How to generate a half-sized version?
![Page 4: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/4.jpg)
Image Sub-Sampling
Throw away every other row and
column to create a 1/2 size image- called image sub-sampling
1/4
1/8
![Page 5: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/5.jpg)
Image Sub-Sampling
1/4 (2x zoom) 1/8 (4x zoom)1/2
![Page 6: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/6.jpg)
Good and Bad Sampling
Good sampling:•Sample often or,•Sample wisely
Bad sampling:•Aliasing!
![Page 7: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/7.jpg)
Aliasing
![Page 8: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/8.jpg)
Alias: n., an assumed name
Picket fence recedinginto the distance willproduce aliasing…
Input signal:
x = 0:.05:5; imagesc(sin((2.^x).*x))
Matlab output:
WHY?
Alias!
Not enough samples
![Page 9: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/9.jpg)
Really bad in video
Wagon-wheel effect
![Page 10: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/10.jpg)
Stroboscopic effect
![Page 11: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/11.jpg)
Sampling Theorem
Continuous signal:
Shah function (Impulse train):
Sampled function:
![Page 12: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/12.jpg)
Sampling Theorem
Sampled function:
Only if
Sampling frequency
![Page 13: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/13.jpg)
Nyquist Frequency
If
Aliasing
When can we recover from ?
Only if (Nyquist Frequency)
We can use
Then and
Sampling frequency must be greater than
![Page 14: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/14.jpg)
Sub-Sampling with Gaussian Pre-Filtering
G 1/4
G 1/8
Gaussian 1/2
• Solution: filter the image, then subsample– Filter size should double for each ½ size reduction. Why?
![Page 15: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/15.jpg)
G 1/4 G 1/8Gaussian 1/2
Sub-Sampling with Gaussian Pre-Filtering
![Page 16: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/16.jpg)
Compare with...
1/4 (2x zoom) 1/8 (4x zoom)1/2
![Page 17: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/17.jpg)
Aliasing
![Page 18: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/18.jpg)
Canon D60 (w/ anti-alias filter) Sigma SD9 (w/o anti-alias filter)
From Rick Matthews website, images by Dave Etchells
![Page 19: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/19.jpg)
Image Resampling
• What about arbitrary scale reduction?
• How can we increase the size of the image?
• Recall how a digital image is formed
– It is a discrete point-sampling of a continuous function
– If we could somehow reconstruct the original function, any new image could be generated, at any resolution and scale
1 2 3 4 5
![Page 20: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/20.jpg)
Image Resampling
• So what to do if we don’t know
1 2 3 4 52.5
1
– Answer: guess an approximation– Can be done in a principled way: filtering
![Page 21: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/21.jpg)
Resampling filters
• What does the 2D version of this hat function look like?
• Better filters give better resampled images– Bicubic is common choice
performs linear interpolation
performs bilinear interpolation
![Page 22: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/22.jpg)
Bilinear interpolation
• A common method for resampling images
![Page 23: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/23.jpg)
Image Rotation
![Page 24: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/24.jpg)
Multi-Resolution Image Representation
• Fourier domain tells us “what” (frequencies, sharpness, texture properties), but not “where”.
• Spatial domain tells us “where” (pixel location) but not “what”.
• We want a image representation that gives a local description of image “events” – what is happening where.
• Naturally, think about representing images across varying scales.
![Page 25: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/25.jpg)
Figure from David Forsyth
![Page 26: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/26.jpg)
Multi-resolution Image Pyramids
High resolution
Low resolution
![Page 27: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/27.jpg)
Space Required for Pyramids
![Page 28: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/28.jpg)
![Page 29: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/29.jpg)
Constructing a pyramid by taking every second pixel leads to layers that badly misrepresent the top layer
![Page 30: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/30.jpg)
Even worse for synthetic images
![Page 31: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/31.jpg)
Decimation
![Page 32: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/32.jpg)
Expansion
![Page 33: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/33.jpg)
Interpolation Results
![Page 34: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/34.jpg)
The Gaussian Pyramid
• Smooth with Gaussians because– a Gaussian*Gaussian=another Gaussian
• Synthesis – smooth and downsample
• Gaussians are low pass filters, so repetition is redundant• Kernel width doubles with each level
![Page 35: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/35.jpg)
2)*( 23 gaussianGG
1G
The Gaussian Pyramid
High resolution
Low resolution
Image0G
2)*( 01 gaussianGG
2)*( 12 gaussianGG
2)*( 34 gaussianGG
blur
blur
blur
down-sample
down-sample
down-sampleblurdown-sample
![Page 36: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/36.jpg)
![Page 37: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/37.jpg)
![Page 38: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/38.jpg)
Pyramids at Same Resolution
![Page 39: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/39.jpg)
expand
expand
expand
Gaussian Pyramid Laplacian Pyramid
The Laplacian Pyramid
0G
1G
2GnG
- =
0L
- =1L
- = 2Lnn GL
)expand( 1 iii GGL
)expand( 1 iii GLG
![Page 40: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/40.jpg)
![Page 41: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/41.jpg)
Applications of Image Pyramids
• Coarse-to-Fine strategies for computational efficiency.• Search for correspondence
– look at coarse scales, then refine with finer scales
• Edge tracking– a “good” edge at a fine scale has parents at a coarser scale
• Control of detail and computational cost in matching– e.g. finding stripes– very important in texture representation
• Image Blending and Mosaicing • Data compression (laplacian pyramid)
![Page 42: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/42.jpg)
Fast Template Matching
![Page 43: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/43.jpg)
Fast Template Matching
![Page 44: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/44.jpg)
![Page 45: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/45.jpg)
![Page 46: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/46.jpg)
![Page 47: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/47.jpg)
![Page 48: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/48.jpg)
![Page 49: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/49.jpg)
Blending Apples and Oranges
![Page 50: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/50.jpg)
Image Blending and Mosaicing
![Page 51: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/51.jpg)
Pyramid blending of Regions
![Page 52: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/52.jpg)
Horror Photo
© prof. dmartin
![Page 53: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/53.jpg)
Image Fusion
Multi-scale Transform (MST) = Obtain Pyramid from ImageInverse Multi-scale Transform (IMST) = Obtain Image from Pyramid
![Page 54: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/54.jpg)
Multi-Sensor Fusion
![Page 55: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/55.jpg)
Image Compression
![Page 56: Computer Vision Spring 2012 15-385,-685 Instructor: S. Narasimhan Wean 5409 T-R 10:30am – 11:50am](https://reader035.vdocument.in/reader035/viewer/2022062516/56649de55503460f94adda4c/html5/thumbnails/56.jpg)
Next Class
• Edge Detection