csci 497/597p: computer visionwehrwes/courses/csci497p... · 2019. 1. 11. · last time:...
TRANSCRIPT
![Page 1: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/1.jpg)
Filtering and Edge Detection
Scott WehrweinCSCI 497/597P: Computer Vision
![Page 2: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/2.jpg)
Reading
• Szeliski, Chapter 4.2
![Page 3: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/3.jpg)
Announcements
![Page 4: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/4.jpg)
Goals
• Understand practical considerations involved in filtering:• Handling the boundaries• Preserving total brightness
• Learn the basics numpy for array and image processing.• Understand the basics of edge detection:• The sobel operator as an approximation of the image gradient in the presence
of noise.
![Page 5: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/5.jpg)
Last time: Convolution and cross-correlation
• Cross correlation
• Convolution
S[f ](m,n) =kX
i=�k
kX
j=�k
w(i, j)f(m+ i, n+ j)
S[f ](m,n) =kX
i=�k
kX
j=�k
w(i, j)f(m� i, n� j)
S[f ] = w ⌦ f
S[f ] = w ⇤ f
![Page 6: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/6.jpg)
Last time: Convolution and cross-correlation
• Properties• Shift-invariant: a sensible thing to require• Linearity: convenient
• Can be used for smoothing, sharpening• Also main component of CNNs
![Page 7: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/7.jpg)
Boundary conditions
• What if m-i <0?• What if m-i > image size• Assume f is defined for [−∞,∞] in both directions,
just 0 everywhere else• Same for w
(w ⇤ f)(m,n) =kX
i=�k
kX
j=�k
w(i, j)f(m� i, n� j)
(w ⇤ f)(m,n) =1X
i=�1
1X
j=�1w(i, j)f(m� i, n� j)
![Page 8: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/8.jpg)
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
![Page 9: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/9.jpg)
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
![Page 10: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/10.jpg)
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
![Page 11: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/11.jpg)
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
![Page 12: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/12.jpg)
Boundary conditions
90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
![Page 13: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/13.jpg)
Boundary conditions in practice
• “Full convolution”: compute if any part of kernel intersects with image• requires padding• Output size = m+k-1
• “Same convolution”: compute if center of kernel is in image• requires padding• output size = m
• “Valid convolution”: compute only if all of kernel is in image• no padding• output size = m-k+1
![Page 14: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/14.jpg)
“valid”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
![Page 15: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/15.jpg)
“same”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
![Page 16: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/16.jpg)
“full”90 0 0 0 0 0 0 0 0 00 0 0 10 10 10 0 0 0 00 0 10 20 20 20 10 40 0 00 10 20 30 0 20 10 0 0 00 10 0 30 40 30 20 10 0 00 10 20 30 40 30 20 10 0 00 10 20 10 40 30 20 10 0 00 10 20 30 30 20 10 0 0 00 0 10 20 20 0 10 0 20 00 0 0 10 10 10 0 0 0 0
![Page 17: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/17.jpg)
Padding Modes (OpenCV)
• BORDER_REPLICATE: aaaaaa|abcdefgh|hhhhhhh• BORDER_REFLECT: fedcba|abcdefgh|hgfedcb• BORDER_REFLECT_101: gfedcb|abcdefgh|gfedcba• BORDER_WRAP: cdefgh|abcdefgh|abcdefg• BORDER_CONSTANT: iiiiii|abcdefgh|iiiiiii
(with specified constant ‘i’; default = 0)
(image)(L pad) (R pad)
![Page 18: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/18.jpg)
Padding Modes (OpenCV)
• BORDER_REPLICATE: aaaaaa|abcdefgh|hhhhhhh• BORDER_REFLECT: fedcba|abcdefgh|hgfedcb• BORDER_REFLECT_101: gfedcb|abcdefgh|gfedcba• BORDER_WRAP: cdefgh|abcdefgh|abcdefg• BORDER_CONSTANT: iiiiii|abcdefgh|iiiiiii
(with specified constant ‘i’; default = 0)
(image)(L pad) (R pad)
![Page 19: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/19.jpg)
The sum of the kernel• The sum of all kernel weights is important.
*1 1 1
1 1 11 1 1
=
![Page 20: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/20.jpg)
The sum of the kernel• The sum of all kernel weights is important.
*1 1 1
1 1 11 1 1
=1
9
Usually want it to sum to either 1 or 0.
![Page 21: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/21.jpg)
Edge detection
• Convert a 2D image into a set of curves– Extracts salient features of the scene– More compact than pixels
![Page 22: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/22.jpg)
Origin of Edges
• Edges are caused by a variety of factors
depth discontinuity
surface color discontinuity
illumination discontinuity
surface normal discontinuity
![Page 23: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/23.jpg)
Images as functions…
• Edges look like steep cliffs
![Page 24: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/24.jpg)
Characterizing edges• An edge is a place of rapid change in the image
intensity function
imageintensity function
(along horizontal scanline) first derivative
edges correspond toextrema of derivativeSource: L. Lazebnik
![Page 25: CSCI 497/597P: Computer Visionwehrwes/courses/csci497p... · 2019. 1. 11. · Last time: Convolution and cross-correlation •Properties •Shift-invariant: a sensible thing to require](https://reader033.vdocument.in/reader033/viewer/2022060903/609f617bca84e0063226b12f/html5/thumbnails/25.jpg)
• How can we differentiate a digital image F[x,y]?– Option 1: reconstruct a continuous image, f, then
compute the derivative– Option 2: take discrete derivative (finite difference)
1 -1
How would you implement this as a linear filter?
Image derivatives
-1
1: :
Source: S. Seitz