2/6/17

1

Scale Space & DoG

CS 510 Lecture #8

February 3rd, 2017

Announcements

• Read the SIFT paper for Monday– Allocate time: 19 pages is long– Be prepared to answer questions– Be prepared to ask questions about parts you

don’t understand– Focus on attention windows, not descriptors

• I need to talk to– Brad Wirtz, Yashad Samant, Nick Lawrence– Boli Gu, Peter Liu, Christina Yang

2/6/17 2

Back to image information…

• Interpolation introduces high-frequency noise– NN more than BiLinear more than BiCubic

• What about changes in scale?– In particular, reducing the image size?

2/6/17 3

The Nyquist Rate• What if the frequency is above N/2?

– You have fewer than one sample per half-cycle

– High frequencies look like lower frequencies

2/6/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 4

Graphicfrom“Computer Graphics:Principles and Practice” by Foley, van Dam, Feiner & Hughes.

Image Reductions• Anytime the target image has a lower

resolution than the source image, prevent frequency aliasing by low-pass filtering.– In practice, convolve with a Gaussian to

avoid Gibbs ringing– Determine Nyquist rate for target image– Select s– Convolve source image with g(s)– Apply geometric transformation to result

2/6/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 5

Convolution

2/6/17 CS510,ImageComputation,©RossBeveridge&BruceDraper 6

• Formally, convolution is often expressed as follows:

Ø Ofcourse,wearedealingwithfinite,discretefunctions:

2/6/17

2

On a real image – Sigma = 4.0

2/6/17 7

Resolution

• The resolution of an image is the inverse of the spatial area covered by each pixel

• Resolution depends on– The image size of the camera– The field of view of the lens– The distance to the target

• Doubling the distance halves the resolution

2/6/17 8

Scale Space• Appearance is a function of resolution• With decreasing resolution:

– A checkerboard • Starts with black and white pixels• As resolution drops, more pixels become grey• Finally, the whole board is one gray pixel

– A bar • Starts as a stripe• Becomes a thin line• Finally disappears

• The goal of scale space is to simulate this process

2/6/17 9

Best Case: Raw Image

• Model pixels as point-wise intensity estimates– An approximation

• Pixels have area– Pixels don’t overlap

• We ignore blooming– Ignore gaps

• Sensitive to micro-translations• Same model ray tracers use…

2/6/17 10

Scale Space

• Pixels should average response over an area to reduce aliasing

• Begin by convolving source image with G(σ) = 1– Every pixel covers 1 source unit of area

• σ = 1 is the “base resolution”

2/6/17 11

Simulate Lower Scales

• To cut resolution in half, convolve base with G(σ) = 2.

• To cut resolution by a quarter, use G(σ) = 4

• Problem: this is expensive– Lots of image convolutions– The convolution masks keep getting bigger!

2/6/17 12

2/6/17

3

Image Pyramids• Fortunately, lower resolutions require

fewer pixels• When a2 = b2 + c2, 𝐼⨂𝐺 𝑎 = 𝐼⨂𝐺 𝑏 ⨂𝐺 𝑐

• Therefore– start with base image (σ=1)– Convolve with G(σ=2)– (Sub)sample every other pixel– Repeat

2/6/17 13

Image Pyramids

2/6/17 14

Produces a multi-scaleimage representation

Masks never get biggerImages keep shrinking

Pyramid size < 1.5 image size

Imagefromhttps://compvisionlab.wordpress.com/2013/04/28/image-pyramids-theory/

Good News: OpenCV

• OpenCV provides functions for building image pyramids

• It also has a tutorial on the subject…

2/6/17 15

Finally! We’re ready for attention!

• Model of Attention: On-center, off-surround– Image patches are interesting if

• What’s inside the patch looks different from• What’s outside the patch

– Recall Treesman’s Feature Integration Theory• Difference can be defined in:

– Intensity– Color– Edge orientation

2/6/17 16

On-Center, Off-Surround

• How do we measure on-center, off-surround differences?

• Assume we know the target size (σ)– Convolution with G(σ) roughly averages the

center response– Convolution with G(2σ) roughly averages the

surround response– 𝐼⨂𝐺 𝜎 − 𝐼⨂𝐺 2𝜎

2/6/17 17

Visualizing DoG

2/6/17 18

http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/diffgaussians/

2/6/17

4

Visualizing DoG in 2D

2/6/17 19

https://www.researchgate.net/publication/221912155_Surround_Suppression_and_Recurrent_Interactions_V1-V2_for_Natural_Scene_Boundary_Detection/figures?lo=1

SometimescalledaMexicanHatfilter

DoG Across Scales• We don’t know the

target scale• So we compute DoG

across scales!• Every pixel already

has σ = 1 • Pixels in the layer

above have σ = 2• So we subtract layers!

2/6/17 20

Example

• Circles represent attention windows– Size based on 2 σ

• Every window has– A center– A different surround

2/6/17 21

https://computervisionblog.w

ordpress.com/2011/04/25/surf-on-im

ages-feature-point-matching/