subsampling and image pyramids - computer...
TRANSCRIPT
![Page 1: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/1.jpg)
Subsampling and image pyramids
15-463, 15-663, 15-862Computational Photography
Fall 2017, Lecture 5http://graphics.cs.cmu.edu/courses/15-463
![Page 2: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/2.jpg)
Course announcements
Respond to Doodle about rescheduling the September 27th lecture!
- Link available on Piazza.- Currently 11 responses. Only one more since Monday :-( .
![Page 3: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/3.jpg)
Course announcements
• Homework 1 is due tomorrow.- Any remaining questions?- How was it?
• Homework 2 will be posted tonight and will be due two weeks from now.- Much larger than homework 1.- Start early! Experiments take a long time to run.
![Page 4: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/4.jpg)
Overview of today’s lecture
• Finish non-linear filtering: non-local means.
• Image downsampling.
• Aliasing.
• Gaussian image pyramid.
• Laplacian image pyramid.
![Page 5: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/5.jpg)
Slide credits
Most of these slides were adapted directly from:
• Kris Kitani (15-463, Fall 2016).
Some slides were inspired or taken from:
• Bernd Girod (Stanford University).• Steve Marschner (Cornell University).• Steve Seitz (University of Washington).
![Page 6: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/6.jpg)
Image downsampling
![Page 7: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/7.jpg)
This image is too big to fit on the screen. How would you reduce it to half its size?
![Page 8: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/8.jpg)
delete even rowsdelete even columns
1/2
1/4
1/8
Throw away half the rows and columns
What is the problem with this approach?
Naïve image downsampling
delete even rowsdelete even columns
![Page 9: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/9.jpg)
1/2 1/4 (2x zoom) 1/8 (4x zoom)
Naïve image downsampling
What is the 1/8 image so pixelated (and do you know what this effect is called)?
![Page 10: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/10.jpg)
Aliasing
![Page 11: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/11.jpg)
Reminder
Images are a discrete, or sampled, representation of a continuous world
![Page 12: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/12.jpg)
Sampling
Very simple example: a sine wave
How would you discretize this signal?
![Page 13: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/13.jpg)
Sampling
Very simple example: a sine wave
![Page 14: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/14.jpg)
Sampling
Very simple example: a sine wave
Can I take as many samples as I want?
How many samples should I take?
![Page 15: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/15.jpg)
Sampling
Very simple example: a sine wave
Can I take as few samples as I want?
How many samples should I take?
![Page 16: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/16.jpg)
Undersampling
Very simple example: a sine wave
Unsurprising effect: information is lost.
![Page 17: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/17.jpg)
Undersampling
Very simple example: a sine wave
Unsurprising effect: information is lost.
Surprising effect: can confuse the signal with one of lower frequency.
![Page 18: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/18.jpg)
Undersampling
Very simple example: a sine wave
Unsurprising effect: information is lost.
Surprising effect: can confuse the signal with one of lower frequency.
Note: we could always confuse the signal with one of higher frequency.
![Page 19: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/19.jpg)
Aliasing
Fancy term for: Undersampling can disguise a signal as one of a lower frequency
Unsurprising effect: information is lost.
Surprising effect: can confuse the signal with one of lower frequency.
Note: we could always confuse the signal with one of higher frequency.
![Page 20: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/20.jpg)
Aliasing in textures
![Page 21: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/21.jpg)
Aliasing in photographs
This is also known as “moire”
![Page 22: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/22.jpg)
Temporal aliasing
![Page 23: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/23.jpg)
Temporal aliasing
![Page 24: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/24.jpg)
Temporal aliasing
![Page 25: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/25.jpg)
Anti-aliasing
How would you deal with aliasing?
![Page 26: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/26.jpg)
Anti-aliasing
How would you deal with aliasing?
Approach 1: Oversample the signal• This is how camera manufacturers started focusing so heavily on number of
megapixels.
![Page 27: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/27.jpg)
Anti-aliasing in textures
aliasing artifacts anti-aliasing by oversampling
![Page 28: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/28.jpg)
Anti-aliasing
How would you deal with aliasing?
Approach 1: Oversample the signal• This is how camera manufacturers started focusing so heavily on number of
megapixels.
Approach 2: Smooth the signal• Remove some of the high frequency effects that cause aliasing.• Lose information, but better than aliasing artifacts.
How would you smooth a signal?
![Page 29: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/29.jpg)
Anti-aliasing
Question 1: How much smoothing do I need to do to avoid aliasing?
Question 2: How many samples do I need to take to avoid aliasing?
We’ll see what this means in the next lecture.
Answer to both: Enough to reach the Nyquist limit.
![Page 30: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/30.jpg)
Quick aside: optical anti-aliasing
Lenses act as (optical) smoothing filters.
![Page 31: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/31.jpg)
Quick aside: optical anti-aliasing
Lenses act as (optical) smoothing filters.
made of silicon, emits electrons from photons
photodiodephotodiode
silicon for read-out etc. circuitry
color filtercolor filter
helps photodiode collect more light
(also called lenslet)
microlensmicrolens
• Lenslets also filter the image to avoid resolution artifacts.
• Lenslets are problematic when working with coherent light.
• Many modern cameras do not have lenslet arrays.
We will discuss these issues in more detail at a later lecture.potential
wellpotential
well
stores emitted electrons
Slide from lecture 2: Basic imaging sensor design
![Page 32: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/32.jpg)
Quick aside: optical anti-aliasing
Lenses act as (optical) smoothing filters.• Sensors often have a lenslet array in front of them as an anti-aliasing (AA) filter.• However, the AA filter means you also lose resolution.• Nowadays, due the large number of sensor pixels, AA filters are becoming unnecessary.
Photographers often hack their cameras to remove the AA filter, in order to avoid the loss of resolution.
a.k.a. “hot rodding”
![Page 33: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/33.jpg)
Quick aside: optical anti-aliasing
without AA filter with AA filter
Example where AA filter is needed
![Page 34: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/34.jpg)
Quick aside: optical anti-aliasing
without AA filter with AA filter
Example where AA filter is unnecessary
![Page 35: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/35.jpg)
Gaussian filterdelete even rows
delete even columns
1/2
1/4
1/8
Apply a smoothing filter first, then throw away half therows and columns
Better image downsampling
Gaussian filterdelete even rows
delete even columns
![Page 36: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/36.jpg)
1/2 1/4 (2x zoom) 1/8 (4x zoom)
Better image downsampling
![Page 37: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/37.jpg)
1/2 1/4 (2x zoom) 1/8 (4x zoom)
Naïve image downsampling
![Page 38: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/38.jpg)
Gaussian image pyramid
![Page 39: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/39.jpg)
The name of this sequence of subsampled images
Gaussian image pyramid
![Page 40: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/40.jpg)
filter
sample
filter
sample
Constructing a Gaussian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
sample
Question: How much bigger than the original image is the whole pyramid?
![Page 41: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/41.jpg)
filter
sample
filter
sample
Constructing a Gaussian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
sample
Question: How much bigger than the original image is the whole pyramid?
Answer: Just 4/3 times the size of the original image! (How did I come up with this number?)
![Page 42: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/42.jpg)
What happens to the details of the image?
Some properties of the Gaussian pyramid
![Page 43: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/43.jpg)
What happens to the details of the image?• They get smoothed out as we move to
higher levels.
What is preserved at the higher levels?
Some properties of the Gaussian pyramid
![Page 44: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/44.jpg)
What happens to the details of the image?• They get smoothed out as we move to
higher levels.
What is preserved at the higher levels?• Mostly large uniform regions in the
original image.
How would you reconstruct the original image from the image at the upper level?
Some properties of the Gaussian pyramid
![Page 45: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/45.jpg)
What happens to the details of the image?• They get smoothed out as we move to
higher levels.
What is preserved at the higher levels?• Mostly large uniform regions in the
original image.
How would you reconstruct the original image from the image at the upper level?• That’s not possible.
Some properties of the Gaussian pyramid
![Page 46: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/46.jpg)
Blurring is lossy
- =
level 0 residual
What does the residual look like?
level 1 (before downsampling)
![Page 47: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/47.jpg)
Blurring is lossy
- =
level 0 level 1 (before downsampling) residual
Can we make a pyramid that is lossless?
![Page 48: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/48.jpg)
Laplacian image pyramid
![Page 49: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/49.jpg)
At each level, retain the residuals instead of the blurred images themselves.
Can we reconstruct the original image using the pyramid?
Laplacian image pyramid
![Page 50: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/50.jpg)
At each level, retain the residuals instead of the blurred images themselves.
Can we reconstruct the original image using the pyramid?• Yes we can!
What do we need to store to be able to reconstruct the original image?
Laplacian image pyramid
![Page 51: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/51.jpg)
Let’s start by looking at just one level
= +
level 0 residual
Does this mean we need to store both residuals and the blurred copies of the original?
level 1 (upsampled)
![Page 52: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/52.jpg)
Constructing a Laplacian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
compute residual
![Page 53: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/53.jpg)
Constructing a Laplacian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
compute residual
What is this part?
![Page 54: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/54.jpg)
Constructing a Laplacian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
compute residual
What is this part?
![Page 55: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/55.jpg)
What do we need to construct the original image?
![Page 56: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/56.jpg)
(1) residuals
What do we need to construct the original image?
![Page 57: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/57.jpg)
(2) smallest image
(1) residuals
What do we need to construct the original image?
![Page 58: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/58.jpg)
repeat:
upsample
sum with residual
until orig resolution reached
Algorithm
Reconstructing the original image
![Page 59: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/59.jpg)
Gaussian vs Laplacian Pyramid
Shown in opposite order for space.
Which one takes more space to store?
![Page 60: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/60.jpg)
Why is it called a Laplacian pyramid?
![Page 61: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/61.jpg)
As with derivative, we can combine Laplace filtering with Gaussian filtering
Reminder: Laplacian of Gaussian (LoG) filter
Laplacian of Gaussian
output
input
“zero crossings” at edges
![Page 62: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/62.jpg)
- =
-
unit Gaussian Laplacian
Difference of Gaussians approximates the Laplacian
Why is it called a Laplacian pyramid?
![Page 63: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/63.jpg)
Why Reagan?
![Page 64: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/64.jpg)
Why Reagan?
The Laplacian Pyramid as a Compact Image Code (1983)
Peter J. Burt , Edward H. Adelson
Donald Reagan was President when the Laplacian pyramid
was invented
![Page 65: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/65.jpg)
Still used extensively
![Page 66: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/66.jpg)
input image
user-provided maskforeground details enhanced, background details reduced
Still used extensively
![Page 67: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/67.jpg)
Still used extensively
Result from:
Paris et al., “Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid,” SIGGRAPH 2011 and CACM 2015
Why “local”?
![Page 68: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/68.jpg)
Other types of pyramids
Steerable pyramid: At each level keep multiple versions, one for each direction.
Wavelets: Huge area in image processing (see 18-793).
![Page 69: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/69.jpg)
What are image pyramids used for?image blendingmulti-scale
texture mapping
focal stack compositing denoising
multi-scale detection multi-scale registration
image compression
motion magnification
your homework 2
![Page 70: Subsampling and image pyramids - Computer Graphicsgraphics.cs.cmu.edu/courses/15-463/2017_fall/lectures/lecture5.pdf · Course announcements • Homework 1 is due tomorrow. - Any](https://reader033.vdocument.in/reader033/viewer/2022053022/60502f8f1ad46564f9178b03/html5/thumbnails/70.jpg)
References
Basic reading:• Szeliski textbook, Sections 3.5
Additional reading:• Burt and Adelson, “The Laplacian Pyramid as a Compact Image Code,” IEEE ToC 1983.
the original Laplacian pyramid paper• Paris et al., “Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid,” SIGGRAPH 2011
and CACM 2015,great paper on modern uses of the Laplacian pyramid, see also the project websitehttps://people.csail.mit.edu/sparis/publi/2011/siggraph/