![Page 1: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/1.jpg)
Computational Photography and Image ManipulationCMPT 469 / 985, Fall 2019
Week 5
Bilateral Filtering
1
![Page 2: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/2.jpg)
Slide credits
• Slides thanks to Ioannis Gkioulekas along with his acknowledgements.
2
![Page 3: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/3.jpg)
• P4 papers – presentations on October 15, Tuesday• Paris et al., Local Laplacian Filters: Edge-aware Image Processing with a
Laplacian Pyramid, SIGGRAPH 2011• Presentation:
• Discussion:
• Chen et al., Real-time Edge-Aware Image Processing with the Bilateral Grid, SIGGRAPH 2007• Presentation:
• Discussion:
• Aydin et al., Temporally Coherent Tone Mapping of HDR Video, SIGGRAPH Asia 2014• Presentation:
• Discussion:
3
![Page 4: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/4.jpg)
Edge-aware filtering and bilateral filtering
![Page 5: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/5.jpg)
Motivational example
Let’s say I want to reduce the amount of detail in this picture. What can I do?
original
![Page 6: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/6.jpg)
Motivational example
original Gaussian filtering
How to smooth out the details in the image without losing the important edges?
![Page 7: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/7.jpg)
Motivational example
original Gaussian filtering bilateral filtering
![Page 8: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/8.jpg)
The problem with Gaussian filtering
input
Gaussian kernel
*
*
*
output
Why is the output so blurry?
![Page 9: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/9.jpg)
The problem with Gaussian filtering
input
Gaussian kernel
*
*
*
output
Blur kernel averages across edges
![Page 10: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/10.jpg)
The bilateral filtering solution
input
bilateral filter kernel
*
*
*
output
Do not blur if there is an edge! How does it do that?
![Page 11: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/11.jpg)
Bilateral filtering
![Page 12: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/12.jpg)
Bilateral filtering vs Gaussian filtering
Gaussian filtering
Bilateral filteringSpatial weighting: favor nearby pixels
Normalization factor
Intensity range weighting: favor similar pixels
![Page 13: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/13.jpg)
Bilateral filtering vs Gaussian filtering
Gaussian filtering
Bilateral filtering
Smooths everything nearby (even edges)Only depends on spatial distance
Smooths ‘close’ pixels in space and intensityDepends on spatial and intensity distance
![Page 14: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/14.jpg)
Gaussian filtering visualization
Output Gaussian Filter Input
![Page 15: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/15.jpg)
Bilateral filtering visualization
Output Bilateral Filter Input
Spatial range Intensity range
![Page 16: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/16.jpg)
Exploring the bilateral filter parameter space
input
ss = 2
ss = 6
ss = 18
sr = 0.1 sr = 0.25sr =
(Gaussian blur)
![Page 17: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/17.jpg)
Does the bilateral filter respect all edges?
input
bilateral filter kernel
*
*
*
output
![Page 18: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/18.jpg)
Does the bilateral filter respect all edges?
input
bilateral filter kernel
*
*
output
Bilateral filter crosses (and blurs) thin edges.
![Page 19: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/19.jpg)
Denoising
noisy input bilateral filtering median filtering
![Page 20: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/20.jpg)
Contrast enhancement
input sharpening based on bilateral filtering
sharpening based on Gaussian filtering
How would you use Gaussian or bilateral filtering for sharpening?
![Page 21: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/21.jpg)
Photo retouching
![Page 22: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/22.jpg)
Photo retouching
original digital pore removal (aka bilateral filtering)
![Page 23: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/23.jpg)
Before
![Page 24: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/24.jpg)
After
![Page 25: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/25.jpg)
Close-up comparison
original digital pore removal (aka bilateral filtering)
![Page 26: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/26.jpg)
Cartoonization
input cartoon rendition
![Page 27: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/27.jpg)
Cartoonization
How would you create this effect?
![Page 28: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/28.jpg)
Cartoonization
edges from bilaterally filtered image bilaterally filtered image
+ =
cartoon rendition
Note: image cartoonization and abstraction are very active research areas.
![Page 29: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/29.jpg)
Is the bilateral filter:
Shift-invariant?
Linear?
• No.
• No.
Does this have any bad implications?
Separable?
• No.
![Page 30: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/30.jpg)
The bilateral grid
Data structure for fast edge-aware image processing.
![Page 31: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/31.jpg)
Modern edge-aware filtering: local Laplacian pyramids
![Page 32: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/32.jpg)
Modern edge-aware filtering: local Laplacian pyramids
input
texture decrease
texture increase
large texture increase
![Page 33: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/33.jpg)
Tonemapping with edge-aware filtering
![Page 34: Computational Photography and Image Manipulation CMPT 469 ...yagiz/cpim/2019-CPIM-05-Bilateral.pdf · Bilateral Filtering 1. Slide credits •Slides thanks to Ioannis Gkioulekas along](https://reader036.vdocument.in/reader036/viewer/2022090608/605e7abae7143b0fba6bad10/html5/thumbnails/34.jpg)
Tonemapping with edge-aware filtering
bilateral filterlocal Laplacian pyramids