applications of image filters
DESCRIPTION
02/04/10. Applications of Image Filters. Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem. Review: Image filtering. 1. 1. 1. 1. 1. 1. 1. 1. 1. Credit: S. Seitz. Image filtering. 1. 1. 1. 1. 1. 1. 1. 1. 1. Credit: S. Seitz. Image filtering. 1. 1. 1. - PowerPoint PPT PresentationTRANSCRIPT
Applications of Image Filters
Computer VisionCS 543 / ECE 549
University of Illinois
Derek Hoiem
02/04/10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Credit: S. Seitz
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Review: Image filtering111
111
111
],[g
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Image filtering111
111
111
],[g
Credit: S. Seitz
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
],[],[],[,
lnkmglkfnmhlk
[.,.]h[.,.]f
Image filtering111
111
111
],[g
Credit: S. Seitz
Filtering in spatial domain-101
-202
-101
* =
Filtering in frequency domain
FFT
FFT
Inverse FFT
=
Sharpening revisited• What does blurring take away?
original smoothed (5x5)
–
detail
=
sharpened
=
Let’s add it back:
original detail
+ α
Application: Hybrid Images
• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
Application: Hybrid Images
• A. Oliva, A. Torralba, P.G. Schyns, “Hybrid Images,” SIGGRAPH 2006
Today’s class
• How to use filters for– Matching– Denoising– Anti-aliasing
• Image representation with pyramids
• Texture– What is it?– How to represent it?
Matching with filters• Goal: find in image
Matching with filters• Goal: find in image• Method 1: SSD
Input 1- sqrt(SSD) Threshold at 0.8
Matching with filters• Goal: find in image• Method 1: SSD• Method 2: Normalized cross-correlation
Input Normalized X-Correlation Threshold at 0.5
Noise
Gaussian Additive Noise
+ =
Noisy Image
Gaussian noise• Mathematical model: sum of many independent
factors• Assumption: independent, zero-mean noise
Source: M. Hebert
Noise• Salt and pepper
noise: contains random occurrences of black and white pixels
• Impulse noise: contains random occurrences of white pixels
• Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Source: S. Seitz
Denoising
Additive Gaussian Noise
Gaussian Filter
Smoothing with larger standard deviations suppresses noise, but also blurs the image
Reducing Gaussian noise
Source: S. Lazebnik
Reducing salt-and-pepper noise by Gaussian smoothing
3x3 5x5 7x7
Alternative idea: Median filtering• A median filter operates over a window by
selecting the median intensity in the window
• Is median filtering linear?Source: K. Grauman
Median filter• What advantage does median filtering have
over Gaussian filtering?– Robustness to outliers
Source: K. Grauman
Median filterSalt-and-pepper noise Median filtered
Source: M. Hebert
• MATLAB: medfilt2(image, [h w])
Median vs. Gaussian filtering3x3 5x5 7x7
Gaussian
Median
Throw away every other row and column to create a 1/2 size image
Subsampling by a factor of 2
• 1D example (sinewave):
Source: S. Marschner
Aliasing problem
Source: S. Marschner
• 1D example (sinewave):
Aliasing problem
• Sub-sampling may be dangerous….• Characteristic errors may appear:
– “Wagon wheels rolling the wrong way in movies”
– “Checkerboards disintegrate in ray tracing”– “Striped shirts look funny on color television”
Source: D. Forsyth
Aliasing problem
Aliasing in video
Slide by Steve Seitz
Source: A. Efros
Aliasing in graphics
Sampling and aliasing
• When sampling a signal at discrete intervals, the sampling frequency must be 2 fmax;
• fmax = max frequency of the input signal.
• This will allows to reconstruct the original perfectly from the sampled version
good
bad
v v v
Nyquist-Shannon Sampling Theorem
Anti-aliasing
Solutions:• Sample more often
• Get rid of all frequencies that are greater than half the new sampling frequency– Will lose information– But it’s better than aliasing– Apply a smoothing filter
Algorithm for downsampling by factor of 2
1. Start with image(h, w)2. Apply low-pass filter
im_blur = imfilter(image, fspecial(‘gaussian’, 7, 1))
3. Sample every other pixelim_small = im_blur(1:2:end, 1:2:end);
Anti-aliasing
Forsyth and Ponce 2002
Subsampling without pre-filtering
1/4 (2x zoom) 1/8 (4x zoom)1/2
Slide by Steve Seitz
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8Gaussian 1/2
Slide by Steve Seitz
Gaussian pyramid
Source: Forsyth
Laplacian filter
Gaussianunit impulse
Laplacian of Gaussian
Source: Lazebnik
Laplacian pyramid
Source: Forsyth
Computing Gaussian/Laplacian Pyramid
http://sepwww.stanford.edu/~morgan/texturematch/paper_html/node3.html
Can we reconstruct the original from the laplacian pyramid?
Related idea: 2d wavelets
2d Wavelets
Matlab: wavemenu
Image representation
• Pixels: great for spatial resolution, poor access to frequency
• Fourier transform: great for frequency, not for spatial info
• Pyramids/wavelets: balance between spatial and frequency information
Major uses of image pyramids
• Compression
• Object detection– Scale search– Features
• Detecting stable interest points
• Registration– Course-to-fine
Texture
Source: Forsyth
Texture and Material
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
Texture and Orientation
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
Texture and Scale
http://www-cvr.ai.uiuc.edu/ponce_grp/data/texture_database/samples/
What is texture?
• Regular or stochastic patterns caused by bumps, grooves, and/or markings
How can we represent texture?
• Measure frequencies at various orientations and scales
Overcomplete representation: filter banks
LM Filter Bank
Code for filter banks: www.robots.ox.ac.uk/~vgg/research/texclass/filters.html
Filter banks• Process image with each filter and keep
responses (or squared/abs responses)
Representing texture• Idea 1: take simple statistics (e.g., mean, std)
of various absolute filter responses
Can you match the texture to the response?
Mean abs responses
FiltersA
B
C
1
2
3
Representing texture by mean abs response
Mean abs responses
Filters
Representing texture• Idea 2: take vectors of filter responses at each pixel and
cluster them, then take histograms (more on in coming weeks)
Things to remember
• When matching using a filter, normalized cross correlation is preferred
• Use Gaussian or median filter for denoising
• Beware of aliasing – use lowpass filter to downsample
• Laplacian pyramids and wavelets provide spatial/frequency information
• Filter banks provide overcomplete representation, good for modeling/recognizing texture
Next class
• Edges and lines
Questions