lec04 subsample web - cornell university• good (low-cost) links follow intensity edges – want...
TRANSCRIPT
![Page 1: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/1.jpg)
Lecture 4: Image Resampling and Reconstruc6on
CS4670: Computer Vision Kavita Bala
![Page 2: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/2.jpg)
Announcements
• PA 1 is out – Due next Thursday
• Demo info etc. online
• Prelim March 5th (Thu)
![Page 3: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/3.jpg)
PA 1
![Page 4: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/4.jpg)
Image Scissors
• Today’s Readings – Intelligent Scissors, Mortensen et. al, SIGGRAPH 1995
Aging Helen Mirren
![Page 5: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/5.jpg)
Extrac6ng objects
• How can this be done? – hard to do manually
• By selec6ng each pixel on the boundary – hard to do automa6cally (“image segmenta6on”) – preZy easy to do semi-automatically
![Page 6: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/6.jpg)
Image Scissors (with demo!)
![Page 7: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/7.jpg)
Intelligent Scissors • Approach answers basic ques6on
– Q: how to find a path from seed to mouse that follows an object boundary as closely as possible?
– A: define a path that stays as close as possible to edges
![Page 8: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/8.jpg)
Intelligent Scissors • Basic Idea
– Define edge score for each pixel • edge pixels have low cost
– Find lowest cost path from seed to mouse
seed
mouse
Questions • How to define costs? • How to find the path?
![Page 9: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/9.jpg)
Let’s look at this more closely • Treat the image as a graph
Graph • node for every pixel p • link between every adjacent pair of pixels, p,q • cost c for each link Note: each link has a cost • this is a little different than the figure before where
each pixel had a cost
p
q
c
![Page 10: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/10.jpg)
• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link
• c ∝ – |intensity of r – intensity of s|
Defining the costs
Want to hug image edges: how to define cost of a link?
p
q
c r
s
![Page 11: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/11.jpg)
Defining the costs
c can (almost) be computed using a cross-correlation filter • assume it is centered at p
p
q
c r
s
![Page 12: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/12.jpg)
Defining the costs
1
-1
c can (almost) be computed using a cross-correlation filter • assume it is centered at p A couple more modifications • Scale the filter response by length of link c. Why? • Make c positive – Set c = (max-|filter response|)*length – where max = maximum |filter response| over all pixels in the image
p
q
c r
s
w -1 -1
1 1
x
y z
![Page 13: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/13.jpg)
Dijkstra’s shortest path algorithm
0 5 3 1
3 3
4 9
2
Algorithm 1. init node costs to , set p = seed point, cost(p) = 0 2. expand p as follows:
for each of p’s neighbors q that are not expanded set cost(q) = min( cost(p) + cpq, cost(q) )
link costs
![Page 14: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/14.jpg)
Dijkstra’s shortest path algorithm
if q’s cost changed, make q point back to p put q on the ACTIVE list (if not already there)
4
1 0
5
3
3 2 3
9
5 3 1
3 3
4 9
2
1 1
Algorithm 1. init node costs to , set p = seed point, cost(p) = 0 2. expand p as follows:
for each of p’s neighbors q that are not expanded set cost(q) = min( cost(p) + cpq, cost(q) )
![Page 15: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/15.jpg)
4
1 0
5
3
3 2 3
9
5
3 1 3 3
4 9
2
1 5
2 3 3
3 2
4
3. set r = node with minimum cost on the ACTIVE list 4. repeat Step 2 for p = r
Dijkstra’s shortest path algorithm
Algorithm 1. init node costs to , set p = seed point, cost(p) = 0 2. expand p as follows:
for each of p’s neighbors q that are not expanded set cost(q) = min( cost(p) + cpq, cost(q) )
if q’s cost changed, make q point back to p put q on the ACTIVE list (if not already there)
![Page 16: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/16.jpg)
3
1 0
5
3
3 2 3
6
5
3 1 3 3
4 9
2
4
3 1
4
5 2
3 3
3 2
4
Dijkstra’s shortest path algorithm
3. set r = node with minimum cost on the ACTIVE list 4. repeat Step 2 for p = r
Algorithm 1. init node costs to , set p = seed point, cost(p) = 0 2. expand p as follows:
for each of p’s neighbors q that are not expanded set cost(q) = min( cost(p) + cpq, cost(q) )
if q’s cost changed, make q point back to p put q on the ACTIVE list (if not already there)
![Page 17: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/17.jpg)
Dijkstra’s shortest path algorithm
• Proper6es – It computes the minimum cost path from the seed to every node in the graph. This set of minimum paths is represented as a tree
– Running 6me, with N pixels: • O(N2) 6me if you use an ac6ve list • O(N log N) if you use an ac6ve priority queue (heap) • takes frac6on of a second for a typical (640x480) image
– Once this tree is computed once, we can extract the op6mal path from any point to the seed in O(N) 6me.
• it runs in real 6me as the mouse moves – What happens when the user specifies a new seed?
![Page 18: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/18.jpg)
Example Results
Kuan-‐chuan Peng Le Zhang
![Page 19: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/19.jpg)
Ques6ons?
![Page 20: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/20.jpg)
Image Scaling
This image is too big to fit on the screen. How can we generate a half-‐sized version?
Source: S. Seitz
![Page 21: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/21.jpg)
Image sub-‐sampling
1/4 (2x zoom) 1/16 (4x zoom)
Why does this look so crujy?
1/2
Source: S. Seitz
![Page 22: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/22.jpg)
Image sub-‐sampling
Source: F. Durand
![Page 23: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/23.jpg)
What is aliasing?
• What if we “missed” things between the samples? • Simple example: undersampling a sine wave
– unsurprising result: informa6on is lost
– surprising result: indis6nguishable from lower frequency
– also was always indis6nguishable from higher frequencies
– aliasing: signals “traveling in disguise” as other frequencies
23
![Page 24: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/24.jpg)
Aliasing
• To avoid aliasing: – sampling rate ≥ 2 * max frequency in the image
• said another way: ≥ two samples per cycle
– This minimum sampling rate is called the Nyquist rate
Source: L. Zhang
![Page 25: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/25.jpg)
Nyquist limit – 2D example
Good sampling
Bad sampling
![Page 26: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/26.jpg)
Aliasing
• When downsampling by a factor of two – Original image has frequencies that are too high
• How can we fix this?
![Page 27: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/27.jpg)
Gaussian pre-‐filtering
G 1/4
G 1/8
Gaussian 1/2
• Solu6on: filter the image, then subsample
Source: S. Seitz
![Page 28: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/28.jpg)
Subsampling with Gaussian pre-‐filtering
G 1/4 G 1/8 Gaussian 1/2
• Solu6on: filter the image, then subsample
Source: S. Seitz
![Page 29: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/29.jpg)
Compare with...
1/4 (2x zoom) 1/16 (4x zoom) 1/2
Source: S. Seitz
![Page 30: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/30.jpg)
Gaussian pre-‐filtering • Solu6on: filter
the image, then subsample
blur
F0 H *
subsample blur subsample … F1
F1 H *
F2 F0
![Page 31: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/31.jpg)
blur
F0 H *
subsample blur subsample … F1
F1 H *
F2 F0
{ Gaussian pyramid
![Page 32: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/32.jpg)
Gaussian pyramids [Burt and Adelson, 1983]
• In computer graphics, a mip map [Williams, 1983] • A precursor to wavelet transform Gaussian Pyramids have all sorts of applica6ons in computer vision
Source: S. Seitz
![Page 33: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/33.jpg)
Gaussian pyramids [Burt and Adelson, 1983]
• How much space does a Gaussian pyramid take compared to the original image?
Source: S. Seitz
![Page 34: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/34.jpg)
Memory Usage
• What is the size of the pyramid?
34
![Page 35: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/35.jpg)
Gaussian Pyramid
![Page 36: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/36.jpg)
Aside: Laplacian
• Laplacian: divergence of gradient
• 0 at edge
f
h
f * h
![Page 37: lec04 subsample web - Cornell University• good (low-cost) links follow intensity edges – want intensity to change rapidly ⊥ to the link • c ∝ – |intensity of r – intensity](https://reader033.vdocument.in/reader033/viewer/2022051916/600834771e2017729e58b18f/html5/thumbnails/37.jpg)
expand
Gaussian Pyramid Laplacian Pyramid
The Laplacian Pyramid
0G
1G2GnG
- =
0L
- = 1L
- = 2Lnn GL =
)expand( 1+−= iii GGL)expand( 1++= iii GLG