image features & texture analysis
TRANSCRIPT
10/6/10
1
Jesus J Caban
Image Features & Texture Analysis
10/4/10 1
Outline
Project proposal
Review assignment #1
Lecture: Image features
Review requirements for assignment #2
10/4/10 2
10/6/10
2
Assignment #1 Question #3: “What about if you take your own pictures…?”
QR Code 10/4/10 3
Assignment #1 – Programming Part
cv.FindContours mathematical morphology
cv.FitEllipse2 Least squares
RBG vs. GBR
Area / Counting individual pixels might be too noisy
10/4/10 4
10/6/10
3
Image Statistics Estimating statistical properties of an image is important
Classification Description Quantification Quality measurement Forensics Alignment
Statistical properties can be either Global Local
Wang 2006 10/4/10 5
Image Descriptor
… …
• Image descriptor • Feature vector • Features
10/4/10 6
10/6/10
4
Color Mean Descriptor
Idea: obtain the representative color by computing the average intensity within each dimension.
Describes the central tendency of the histogram
10/4/10 7
Color Variance Descriptor
Describes the distance from the mean color
Measures the dispersion of the colors
2nd standard moment
10/4/10 8
10/6/10
5
Other Descriptors
Absolute Deviation
Standard Deviation
Skewness
Kurtosis
10/4/10 9
Additional Statistical Measurements
1) Mean:
3) Absolute Deviation:
2) Variance:
4) Standard Deviation:
6) Kurtosis
5) Skewness
What about more robust color descriptors? 10/4/10 10
10/6/10
6
1. Color Histogram Descriptor
0 1 2 3 4 5 6 7
Gray Value
2 2 4 6 7 8 6 1
Count 0.05 0.05 .11 .17 .20 .22 .17 .03
Rel. Freq.
0 3 3 2 5 5 1 1 0 3 4 5 2 2 2 4 4 4 3 3 4 4 5 5 3 4 5 5 6 6 7 6 6 6 6 5
0
1
2
3
4
5
6
7
8
9
0 1 2 3 4 5 6 7
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
1
0 1 2 3 4 5 6 7
10/4/10 11
Color Histogram The color histogram is easy to compute and effective in
characterizing both the global and local distribution of colors in an image.
Robust to translation and rotation about the view axis and changes only slowly with the scale, occlusion and viewing angle. �
10/4/10 12
10/6/10
7
Different Images can have the same histogram
Images with the same histogram
10/4/10 13
2. Dominant colors
10/4/10 14
10/6/10
8
Dominant Color Descriptor
N is the number of dominant colors
Ci is a vector of corresponding color component
Pi is the normalized percentage / fraction of the pixels in the image corresponding to color ci
Vi describes the variation of the color values within group Ci
S is the spatial coherence that represents the overall spatial homogeneity of the dominant colors. Often zero.
10/4/10 15
Dominant Color Descriptor
10/4/10 16
10/6/10
9
Recall: Histogram is not unique
Images with the same histogram
10/4/10 17
3. Color Structure Descriptor Represent images by:
color distribution local spatial structure of the color
A structuring element is used Image scanned by the structuring element Count the number of times a particular color is contained
within the structuring element
10/4/10 18
10/6/10
10
Color Structure Descriptor - Example
Diagram by C. Royo 10/4/10 19
Color Structure Descriptor - Example
Diagram by C. Royo 10/4/10 20
10/6/10
11
Images have more than just colors
10/4/10 21
Textures
10/4/10 22
10/6/10
12
What’s a texture?
Texture consists of texture primitives or texture elements, sometimes called texels. Texture can be described as fine, coarse, grained, smooth, etc.
Textural features are found in the intensity differences and structure of a texture.
The structure represents the spatial relationship between texels.
How can we capture textural properties? *Texture Analysis* 10/4/10 23
Second-Order Statistics
Second-order statistics Measure the likelihood of observing an intensity
value i and j at an average distance ∆ = (ζx, ζy, ζz)
1 2 N
2
1
N
i
j
i j
45o 0o
• Computed from co-occurrence matrices • Joint probability of pairs of gray levels • Store probability in 2D matrix • Rotational invariant by averaging matrices for different
angles (0o, 45o, 90o, 135o)
10/4/10 24
10/6/10
13
Gray Level Co-occurrence
The GLCM is defined by:
– where nij is the number of occurrences of the pixel values (i,j) lying at distance d in the image.
– The co-occurrence matrix Pd has dimension n× n, where n is the number of gray levels in the image.
1 2 N
2
1
N
i
j
10/4/10 25
Gray Level Co-occurrence
Ng = 8 D = 1 0 degree direction
10/4/10 26
10/6/10
14
Gray Level Co-occurrence
Ng = 8 D = 1 45 -degree direction
10/4/10 27
Gray Level Co-occurrence
Compute four matrices: 0◦, 45◦, 90◦, 135◦ Combine the four matrices and estimate the average values for each position
pij
10/4/10 28
10/6/10
15
Co-occurrence features
Haralick et. al., proposed 14 texture features.
Most widely used are 1. Contrast 2. Correlation 3. Homogeneity 4. Energy 5. Entropy
10/4/10 29
Co-occurrence features
Angular second moment (Uniformity)
Contrast
Correlation
10/4/10 30
10/6/10
16
Co-occurrence features
Sum of Squares
Inverse Difference Moments
Sum Average
10/4/10 31
Co-occurrence features
Entropy
10/4/10 32
10/6/10
17
Co-occurrence features - Applications
10/4/10 33
Higher-Order Statistics Run-length matrices
Finds gray-level runs within the volume
Consecutive pixels that have the same gray level intensity along a specific linear orientation
Matrix p(i, r) is defined as the number of pixels of gray level i and run length r along a direction d = {0o, 45o, 90o, 135o }
i r
1 2 N
2
1
N
i
r
j
10/4/10 34
10/6/10
18
Run-length Features
Short Run Emphasis
Long Run Emphasis
10/4/10 35
Run-length Features Gray Level Nonuniformity
Run length Nonuniformity
10/4/10 36
10/6/10
19
Other Run-length Features
10/4/10 37
Combine Textural Features
… …
10/4/10 38
10/6/10
20
Additional Image Features?
What additional image features should we capture?
10/4/10 39
Shape Descriptors
Edges in images constitute an important feature
Human eyes are sensitive to edge features for image perception
Histograms are commonly used to represent global features
There are two groups of shape descriptors 1. Contour-based Shape Descriptor 2. Region-based Shape Descriptor
Shape Descriptors from MPEG 7 10/4/10 40
10/6/10
21
Edge Histogram Descriptor
Divide image into 4x4 nonoverlapping block Generate a histogram of edge distribution for each sub-image Edges in the sub-image are categorized into 5 types:
0o, 45o, 90o, 135o and non-directional
Therefore, 5 x 16 = 80-histogram bins
10/4/10 41
Edge Histogram Descriptor
Do we need 16 blocks or ROIs?
10/4/10 42
10/6/10
22
Shape Descriptors 1. D2 Shape Distribution:
A histogram of distances between pairs of points on the surface
2. Extended Gaussian Image: A spherical function giving the distribution of surface normals
3. Shape Histogram (SHELLS): A histogram of distances from the center of mass to points on the surface
4. Spherical Harmonics Descriptor: a collection of spherical function giving the maximal distance from center
of mass as a function of spherical angle and radius.
5. Contour Fourier Descriptors 10/4/10 43
Summary
Diagram by C. Royo 10/4/10 44
10/6/10
23
Limitations of some shape descriptors
Compared with color and texture features, shape features are usually described after images have been segmented into regions or objects.
What about occlusion?
How many points do you need?
10/4/10 45
Thumbnail Image Feature
10/4/10 46
10/6/10
24
Patches as Descriptors
10/4/10 47
10/4/10 48
10/6/10
25
Patches – What do we need? Strong features Invariant to changes
Rotation Scale Affine and perspective transformation Occlusion Lighting
10/4/10 49
Scale space Multi-scale representation of images
Note that the standard deviation of the distribution: sqrt(t)
Details which are smaller than sqrt(t) are mostly removed
10/4/10 50
10/6/10
26
Scale-Space Extrema Detection
10/4/10 51
10/4/10 52
10/6/10
27
1) Extract Features
Run program: $./extract_features image13.jpg �
Output: "image13.jpg,13,0.43,0.2,150,…. �
• At least 25 features • At least capture two different characteristics (color, shape, texture, edge, size, wavelets, etc…)
What?
10/4/10 53
2) CSV file with features
In Linux, Mac, Cygwin, etc… $for f in *.jpg; do ./extract_features $f >> all_features.csv; done�
How?
10/4/10 54
10/6/10
28
3) Pick 12 closest images
Given an input image I, pick 12 closest images Program:
$./closest_images 6 all_features.csv
Output: [image100.jpg] == image03.jpg image43.jpg ….
In Linux, Mac, Cygwin, etc… (ImageMagick) $montage image00.jpg image01.jpg …. results1.jpg �
How?
10/4/10 55