lectures 3: image transforms
DESCRIPTION
Lectures 3: Image Transforms. Professor Heikki Kälviäinen Machine Vision and Pattern Recognition Laboratory Department of Information Technology Faculty of Technology Management Lappeenranta University of Technology (LUT) [email protected] http://www.lut.fi/~kalviai - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/1.jpg)
Machine Vision and Dig. Image Analysis
1 Prof. Heikki Kälviäinen CT50A6100
Lectures 3:Image Transforms
Professor Heikki Kälviäinen
Machine Vision and Pattern Recognition Laboratory Department of Information Technology
Faculty of Technology ManagementLappeenranta University of Technology (LUT)
[email protected]://www.lut.fi/~kalviai
http://www.it.lut.fi/ip/research/mvpr/
![Page 2: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/2.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
2
Content
• Fourier Transform. – Discrete Fourier Transform. – Properties of 2-D Fourier Transform.
• Separability, periodicity, translation, rotation, scaling, convolution, correlation.
– Fast Fourier Transform. – Fourier Transform for image processing and analysis.
• Gabor filtering. – 2-D Gabor filter.– Applications.
• Other transforms.
![Page 3: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/3.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
3
Image Transforms
Fourier TransformCosineSineHadamardHaarSlantKarhunen-LoeveFast KLSinusoidal SDVetc.
![Page 4: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/4.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
4
Fourier Transform
• Jean Baptiste Joseph Fourier. • In 1807:
Any periodic function can be expressed as the sum of sines and/or cosines of different frequencies, each multiplied by a different coefficient. • The sum is nowadays called a Fourier series.• An image is a 2D signal.
From: R.C. Gonzalez and R.E. Woods: Digital Image Processing, 3rd edition, 2008.
![Page 5: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/5.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
5
![Page 6: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/6.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
6
![Page 7: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/7.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
7
Images and Their Fourier Spectra
i = sqrt(-1)F(k) also denoted F(u)
F(x,y) also denoted F(u,v)
![Page 8: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/8.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
8
![Page 9: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/9.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
9
Discrete Fourier Transform (DFT)
![Page 10: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/10.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
10
Properties of 2-D Fourier Transform
• Separability.– 2-D can be computed as series of 1-D.
• Periodicity.• Translation, rotation, and scaling.
– Invariant features. • Convolution and correlation.
– Spatial filtering in the frequency domain. • Etc.
![Page 11: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/11.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
11
![Page 12: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/12.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
12
![Page 13: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/13.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
13
![Page 14: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/14.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
14
![Page 15: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/15.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
15
Fast Fourier Transform (FFT)
• Brute-force implementation of Fourier Transform requires on the order of (MN)^2 summations and additions.
• 1024 x 1024 pixels => the order of a trillion multiplications and summations for one DFT
=> computationally too heavy. • FFT => the order of MN log_2 MN. • Based on the successive doubling method.
![Page 16: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/16.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
16
![Page 17: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/17.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
17
![Page 18: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/18.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
18
![Page 19: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/19.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
19
Fourier Transform for Image Processing and Analysis
Properties of Fourier Transform offer for example the following use:• Feature extraction.
– Frequency domain features. • Image compression. • Image filtering.
– Image enhancement in the frequency domain. – Preprocessing.
![Page 20: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/20.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
20
Fourier Transform: Feature Extraction
• Fourier transforms of texture.• Regular patterns. • Feature extraction.
![Page 21: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/21.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
21
Fourier Transform: Image Compression
Radius (pixels) % Image power
8 95
16 97
32 98
64 99.4
128 99.8
Not so many coefficients needed => image compression.
Lossy/lossless compression?
![Page 22: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/22.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
22
Fourier Transform: Image Filtering
• Enhancement in the frequency domain. • Convolution: f(x)*g(x) F(u) G(u).• Ideal low pass filter: Original (left) and filtered image (right).
![Page 23: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/23.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
23
Enhancement in the Frequency Domain
• Ideal high pass filter:– Original (left) and filtered image (right).
![Page 24: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/24.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
24
Image Processing Using Gabor Filtering
• For local and global feature extraction. • Filtering in time (spatial) space and frequency space.• For image processing and analysis two important parameters:
– Frequency f. – Orientation theta.
• Application example: – Face detection and recognition.– FACEDETECT project (http://www.it.lut.fi/project/facedetect/):
• Machine Vision and Pattern Recognition Laboratory (MVPR), Department of Information Technology, LUT, Finland.
• Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK.
![Page 25: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/25.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
25
Feature Detector: 2-D Gabor Filter
cossin'
sincos'
),( '2''22
2
22
2
2
yxy
yxx
eef
yx fxjy
fx
f
![Page 26: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/26.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
26
Gabor Features
• Maximal joint localization in the spatial and frequency domain.• Smooth and noise tolerant.• Parameters for invariance manipulation:Frequency Envelope sharpness Orientation
![Page 27: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/27.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
27
Constructing Response Matrix
Filter response r(x,y; f,) can be calculated for variousfrequencies f and orientations to construct a response matrix.
columns represent orientationsrows represent frequenciesimage rotation appears as acircular shift of the columns
image scaling appears as ashift of the rows (highfrequencies may vanish)
A SCALE AND ROTATION INVARIANTTREATMENT OF THE RESPONSE MATRIXCAN BE ESTABLISHED, AND THUS, WECAN CONCENTRATE ONLY HOW TOCLASSIFY THEM IN THE STANDARD POSE
![Page 28: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/28.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
28
2-D Gabor Features
discrete frequency [u]
dis
cre
te fr
eq
ue
ncy
[v]
-1/2 -1/4 0 1/4 1/2
-1/2
-1/4
0
1/4
1/2
What do they ”see”?
![Page 29: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/29.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
29
Face verification (authentication) Validating a claimed identity based on the image of a face: are you Mr./Ms. X?
Face recognition (identification)Identifying a person based on an image of his/her face: who are you?
Face detection/localizationLocation of human faces in images at different positions, scales, orientations, and lighting conditions.
Face Detection and Recognition
![Page 30: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/30.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
30
Proposed Algorithm
• Avoiding a scanning window.• Using feature detectors.• Shape-free texture model for the final decision.
![Page 31: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/31.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
31
Evidence Extraction
Requirements
• Scale invariant extraction.• Rotation invariant extraction.• Provides sufficiently small amount of correct candidate points. (n best points from each class; needs confidence measure).
Preferred
• Estimation of evidence scale and orientation.• Fast extraction (scalability).
![Page 32: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/32.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
32
Classifier Construction
eye
eye
nostrilnostril
eyeeye
Gaussian mixturemodel densities(EM estimation)
• Stability property guarantees approximately the Gaussian form of classes in the feature space.
• One class may still consist of several sub-clusters (open eye, closed eye, etc.).
Bayesianclassificationof features
![Page 33: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/33.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
33
Affine Learned Correspondences
Aligned images of objects andmanually selected features Variability and correspondences
1 2
3 4
5 6
![Page 34: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/34.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
34
Affine Hypothesis Search
2
2
3 11 12
4
5
2
2
1
1. Evidence extraction.
2. Affine search and match to correspon- dence model.
Instanceapproved
False alarms occur and hypothesisverification is needed
![Page 35: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/35.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
35
Face Space
• Normalization of space where shape variations and capture effects are removed from patterns.
• Based on three points on the face -> affine registration.
• Optimal with regard to the photometric variance over a big set of faces.
![Page 36: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/36.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
36
Features & Feature Detectors
• Features = salient parts of face.
• Small localization variance and frequent occurrence over population.
• Illumination, scale, rotation, and translation invariance.
• Automatic analysis using the face space desirable.
![Page 37: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/37.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
37
![Page 38: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/38.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
38
![Page 39: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/39.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
39
![Page 40: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/40.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
40
![Page 41: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/41.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
41
![Page 42: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/42.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
42
Other transforms
• DFT, Cosine, Sine, Hadamar, Haar, Slant, Karhunen-Loeve, Fast KL, Sinusoidal transform, SVD transform.
• Basis images: Haar (wavelets) (left), Hadamard (right).– Haar values: positive-black, negative-white, zero-gray– Hadamard values: one – black, minus one – white.
![Page 43: Lectures 3: Image Transforms](https://reader036.vdocument.in/reader036/viewer/2022081417/56815b61550346895dc9486d/html5/thumbnails/43.jpg)
Machine Vision and Dig. Image Analysis
Prof. Heikki Kälviäinen CT50A6100
43
Summary
• Image transforms from the spatial domain into the frequency domain.• Fourier Transform.
– For overall image processing and analysis. – Feature extraction, image compression, and image filtering
(enhancement). – Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT). – Properties of 2-D Fourier Transform.
• Separability, periodicity, translation, rotation, scaling, convolution, correlation.
• Gabor filtering. – For local and global feature extraction. – Orientation and frequency.– 2-D Gabor filter.
• Other transforms. – For example, Cosine Transform for image compression. – More detailed information later in the course.