agenda
DESCRIPTION
Agenda. Project 2- Due this Thursday Office Hours Wed 10:30-12 Image blending Background Constrained optimization. Recall: goal. Formulation: find the best patch f. Given vector field v (pasted gradient), find the value of f in unknown region that optimize: . Pasted gradient. Mask. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/1.jpg)
Agenda
• Project 2- Due this Thursday• Office Hours Wed 10:30-12• Image blending• Background– Constrained optimization
![Page 2: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/2.jpg)
Recall: goal
![Page 3: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/3.jpg)
Formulation: find the best patch f
• Given vector field v (pasted gradient), find the value of f in unknown region that optimize:
Pasted gradient Mask
Background
unknownregion
![Page 4: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/4.jpg)
Notation• Destination image: f* (table)• Source image: g (table)• Output image: f (table)• W: list of (i,j) pixel coordinates from f* we want to replace• dW: list of (i,j) pixel coordinates on border of W• We’ll use p = (i,j) to denote a pixel location
– gp is a pixel value at p = (i,j) from source image,
– fW is the set of pixels we’re trying to find
![Page 5: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/5.jpg)
Notation• Destination image: f* (table)• Source image: g (table)• Output image: f (table)• W: set of (i,j) pixel coordinates from f we want to replace (list of pairs)• dW: set of (i,j) pixel coordinates on border of W (list of pairs) • We’ll use p = (i,j) to denote a pixel location
– gp is a pixel value at p = (i,j) from source image,
– fW is the set of pixels we’re trying to find
With constraint that, for p in dW
sum over all pairs of neighbors in W
![Page 6: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/6.jpg)
Optimization
What is optimal fW without above constraint?
What is known versus unknown?
Variational formulation of solution:The best patch is the one that produces the lowest score, subject to the constraint
Drop subscript
for all p in dOmega
![Page 7: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/7.jpg)
Optimization
Pretend constraint wasn’t there: how to find lowest scoring fW?
1) Brute-force search-Keep guessing different patches f and score them
-Output the best-scoring one
2) Gradient descent-Guess a patch f. Update guess with f = f -
![Page 8: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/8.jpg)
How to estimate gradient?
In general, we can always do it numerically
For above quadratic function, we can calculate in closed form
![Page 9: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/9.jpg)
How to estimate gradient?
In general, we can always do it numerically
For above quadratic function, we can calculate in closed form
![Page 10: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/10.jpg)
Constrained optimization
1) Brute-force search-Keep guessing different patches f and score them
-Output the best-scoring one
2) Gradient descent-Guess a patch f. Update guess with f = f -
What happens when gradient is zero?
![Page 11: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/11.jpg)
Optimization
1) Brute-force search-Keep guessing different patches f and score them
-Output the best-scoring one
2) Gradient descent-Guess a patch f. Update guess with f = f –
3) Closed-form solution (for simple functions)
![Page 12: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/12.jpg)
Constrained optimization
How to handle constraints?
1) Brute-force search-Keep guessing different patches f and score them
-Output the best-scoring one
2) Gradient descent-Guess a patch f. Update guess with f = f -
Correct fp = f*p after a gradient update
![Page 13: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/13.jpg)
Constrained optimization
How to handle constraints?
1) Brute-force search-Keep guessing different patches f and score them
-Output the best-scoring one
2) Gradient descent-Guess a patch f. Update guess with f = f -
What happens when gradient is zero?
![Page 14: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/14.jpg)
Lagrangian optimization
• If there was no constraint, we’d have a closed-form solution
• Is there a way to get closed-form solutions using the constraint?
![Page 15: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/15.jpg)
Lagrangian optimizationmin f(x,y) such that g(x,y) = 0
Imagine we want to synthesize a “two-pixel” patch
![Page 16: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/16.jpg)
Lagrangian optimizationmin f(x,y) such that g(x,y) = 0
and g(x,y) = 0
![Page 17: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/17.jpg)
Write conditions with single equation(just for convenience)
At minimum of F, the its gradient is 0
Therefore, the following conditions hold
![Page 18: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/18.jpg)
Multiple constraintsmin f(x,y) such that g1(x,y) = 0, g2(x,y) = 0
What is f(x,y) in our case? g1(x,y)?
![Page 19: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/19.jpg)
Lagrangian optimization
for p in dW (border pixels)
for all other p in W
Since S is quadratic in f, the above yeilds a set of linear equationsAf =b
f = inv(A)b
![Page 20: Agenda](https://reader035.vdocument.in/reader035/viewer/2022062811/5681617c550346895dd10bb6/html5/thumbnails/20.jpg)