introduction to linear image processing 1 introduction to...
TRANSCRIPT
1 Introduction to Linear Image Processing
IPAM - UCLA July 22, 2013
Introduction to Linear Image Processing
Iasonas Kokkinos
Center for Visual Computing
Ecole Centrale Paris / INRIA Saclay
2 Introduction to Linear Image Processing
dA
dA’
Computer Vision
Image to Symbols
Computer Graphics
Symbols to Image
Imaging
Physics to Image
Image Processing
Image to Image
Image Sciences in a nutshell
3 Introduction to Linear Image Processing
Images as functions
Continuous d=1: Gray
d=3: Color
Discrete
4 Introduction to Linear Image Processing
Image Denoising
5 Introduction to Linear Image Processing
Image Denoising
Key assumption: clean image is smooth
6 Introduction to Linear Image Processing
Moving Average in 2D
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Slide Source: S. Seitz
7 Introduction to Linear Image Processing
Moving Average in 2D
0 10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Slide Source: S. Seitz
8 Introduction to Linear Image Processing
Moving Average in 2D
0 10 20
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Slide Source: S. Seitz
9 Introduction to Linear Image Processing
Moving Average in 2D
0 10 20 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Slide Source: S. Seitz
10 Introduction to Linear Image Processing
Moving Average in 2D
0 10 20 30 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Slide Source: S. Seitz
11 Introduction to Linear Image Processing
Moving Average in 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10
10 10 10 0 0 0 0 0
Slide Source: S. Seitz
12 Introduction to Linear Image Processing
Denoising: input
13 Introduction to Linear Image Processing
Denoising: first application of averaring filter
14 Introduction to Linear Image Processing
Denoising: tenth application of denoising filter
15 Introduction to Linear Image Processing
Denoising: application of larger box filter
16 Introduction to Linear Image Processing
Weighted averaging
17 Introduction to Linear Image Processing
σ = 2 σ = 5
Weighting kernel
σ = 10
Standard deviation, σ: determines spatial support
Gaussian function:
18 Introduction to Linear Image Processing
Moving average
19 Introduction to Linear Image Processing
Gaussian blur
20 Introduction to Linear Image Processing
Image Processing
image image filter
21 Introduction to Linear Image Processing
Linearity
Translation Invariance
Linear, Translation-Invariant (LTI) system
Linear Image Processing
22 Introduction to Linear Image Processing
Linear Image Processing
image image filter
From time-invariance: useful bases.
23 Introduction to Linear Image Processing
Linear Image Processing
image image filter
From time-invariance: useful bases.
24 Introduction to Linear Image Processing
Linear algebra reminder
Orthonormal basis:
Expansion coefficients:
Basis: N linearly independent vectors
Expansion on basis:
Expansion:
25 Introduction to Linear Image Processing
Canonical basis
26 Introduction to Linear Image Processing
Canonical basis for 2D signals
Kronecker delta
27 Introduction to Linear Image Processing
Canonical basis for 2D signals
Kronecker delta
28 Introduction to Linear Image Processing
Canonical basis for 2D signals
Kronecker delta
29 Introduction to Linear Image Processing
Canonical basis for signals: expansion
Unit sample function
Signal expansion:
Identify terms:
Rewrite:
Sifting property:
30 Introduction to Linear Image Processing
Canonical basis for signals and LTI filters
Any signal:
impulse response
Translation-invariance
Output of any LSI filter for any input:
convolution of input with filter’s impulse response
Convolution sum
unit
sample
By linearity:
31 Introduction to Linear Image Processing
Convolution – discrete and continuous
2D convolution sum:
2D convolution integral:
32 Introduction to Linear Image Processing
Linear Image Processing
image image filter
From time-invariance: useful bases.
33 Introduction to Linear Image Processing
Associative Property:
Associative property & efficiency
Separability of Gaussian:
Slow Fast
34 Introduction to Linear Image Processing
Associative Property:
Associative property & accuracy
Derivative of Gaussian:
approximate exact
35 Introduction to Linear Image Processing
Associative Property:
Associative property & multi-scale processing
Semi-group property of Gaussian:
36 Introduction to Linear Image Processing
Denoising: first application of averaging kernel
37 Introduction to Linear Image Processing
Denoising: 10th application of denoising kernel
38 Introduction to Linear Image Processing
Distributive property & efficiency
Distributive property:
Steerable fliter:
W. Freeman and E. Adelson, ‘The design and use of steerable filters’, PAMI, 1991
39 Introduction to Linear Image Processing
Linear algebra reminder: eigenvectors
Eigenvectors:
Full-rank, real and symmetric: eigenbasis
40 Introduction to Linear Image Processing
Eigenvectors and eigenfunctions
Eigenvector:
Eigenfunction:
Input:
Output:
41 Introduction to Linear Image Processing
Eigenfunctions for LTI filters
LTI filter:
Let’s guess:
It works:
Frequency response:
42 Introduction to Linear Image Processing
Expansion on harmonic basis
From orthonormality:
Inner product for complex functions:
Discrete-time:
Continuous-time:
43 Introduction to Linear Image Processing
Change of basis
Canonical expansion:
Eigenbasis expansion:
Rotation matrix from eigenbasis:
Fourier transform: change of basis Rotation from canonical basis to eigenfunction basis
44 Introduction to Linear Image Processing
Fourier Analysis
45 Introduction to Linear Image Processing
Fourier synthesis equation
Continuous-time:
Discrete-time:
46 Introduction to Linear Image Processing
Convolution theorem of Fourier transform
Input expansion:
Output:
Expansions:
47 Introduction to Linear Image Processing
Linear Image Processing
image image filter
From time-invariance: useful bases.
48 Introduction to Linear Image Processing
Convolution theorem
Fourier Analysis Fourier Synthesis
49 Introduction to Linear Image Processing
Convolution theorem and efficiency
Fast Fourier Transform Fast Fourier Transform
50 Introduction to Linear Image Processing
Gaussian blur Time
Frequency
51 Introduction to Linear Image Processing
Moving average Time
Frequency
52 Introduction to Linear Image Processing
Modulation property and Gabor filters
Modulation property:
Gaussian:
Time Frequency
53 Introduction to Linear Image Processing
Modulation property and Gabor filters
Modulation property:
Gaussian:
Gabor:
Time Frequency
54 Introduction to Linear Image Processing
• Consider many combinations of and
Frequency responce
isocurves
Increasing
Increasing
2D Gabor filterbank
55 Introduction to Linear Image Processing
2D Gabor filterbank and texture analysis
56 Introduction to Linear Image Processing
2D Gabor filterbank and texture analysis
57 Introduction to Linear Image Processing
Thursday’s lecture: Pyramids, Scale-Invariant Blobs/Ridges, SIFT,
HOG, Log-polar features, Harmonic analysis on surfaces…
• Linear Time-Invariant filters
• Convolution
• Fourier Transform
• (Derivative-of) Gaussian filters
• Steerable filters
• Gabor filters
Summary
Further reading:
Fast recursive filters: Recursively implementing the Gaussian and its Derivatives - R. Deriche, 1993
Recursive implementation of the Gaussian filter. I. Young, L. Vliet, 1995
Fast IIR Isotropic 2D Complex Gabor Filters with Boundary Initialization,
A Bernardino, J. Santos-Victor, TIP, 2006
Wavelets: A Wavelet Tour of Signal Processing, S. Mallat, 2008