lecture #26 - case western reserve...
TRANSCRIPT
EECS490: Digital Image Processing
Lecture #26
• Moments; invariant moments
• Eigenvector, principal component analysis
• Boundary coding
• Image primitives
• Image representation: trees, graphs
• Object recognition and classes
• Minimum distance classifiers
• Correlation
• Statistical pattern recognition; Bayes decision rule
• Neural network classifiers
EECS490: Digital Image Processing
Moments
mpq = x pyq f x, y( )dxdy++
The calculation of moments of for images andobjects in images is very similar to that used tocalculate statistical moments
mpq = xipyj
q f xi , yj( )j=0
L 1
i=0
L 1
For an LxL discrete image we can write
EECS490: Digital Image Processing
Moments
We can also define central moments in a similar manner
μpq = xi x( )pyj y( )
qf xi , yj( )
j=0
L 1
i=0
L 1
μ00 = m00 = f xi , yj( )j=0
L 1
i=0
L 1
m10 = xi f xi , yj( )j=0
L 1
i=0
L 1
m01 = yi f xi , yj( )j=0
L 1
i=0
L 1
x =m10
m00
y =m01
m00
Some simple moments and means
EECS490: Digital Image Processing
Moments
We can use these second moments to define the firstinvariant moment
μ20 = xi xi( )2f xi , yj( )
j=0
L 1
i=0
L 1
= m20 xm10
20 =μ20
μ002
Computing higher moments and means
μ02 = yj yj( )2f xi , yj( )
j=0
L 1
i=0
L 1
= m02 ym01
02 =μ02
μ002
1 = 20 + 02
EECS490: Digital Image Processing
Invariant Moments
1 = 20 + 02
2 = 20 02( )2+ 4 11
2
3 = 30 3 12( )2+ 21 3 03( )
2
7 = 3 21 03( ) 30 + 12( ) 30 + 12( )23 21 + 03( )
2
+ 3 12 30( ) 21 + 03( ) 3 30 + 12( )2
21 + 03( )2
EECS490: Digital Image Processing
Moments
Reduced 50% insize
Rotated 2˚
Mirrored
Rotated 45˚
Original
EECS490: Digital Image Processing
There are seven moments which are invariant totranslation, rotation, and scale.
Each row in this table should remain constant.The constancy shows the utility of the moment
Moments
EECS490: Digital Image Processing
Moments
© 2002 R. C. Gonzalez & R. E. Woods
Reduced50% insize
Rotated90˚
Translated
Rotated 45˚
Original
Mirrored
EECS490: Digital Image Processing
Representation and Description
© 2002 R. C. Gonzalez & R. E. Woods
Each row in this table should also remainconstant. The constancy shows the utility of themoment
This example is much better than the examplefrom the 2nd edition
EECS490: Digital Image Processing
x =
x1x2x3
=
R
G
B
x =
x1x2
xn
Eigenvectors and Eigenvalues
For RGB images (with 3 components)we can write each pixel as x
For registered multi-spectral images(with n components) we can write x as
n=6 components
EECS490: Digital Image Processing
For these n registered images we can compute themean vector (nx1) and covariance matrix (nxn) forthe set of all x
x =
x1x2
xn
mx = E x{ } =
E x1{ }E x2{ }
E xn{ }
Cx = E x mx( ) x mx( )T{ } =
E x1 m1( ) x1 m1( ){ } E x1 m1( ) x2 m2( ){ } E x1 m1( ) xn mn( ){ }E x2 m2( ) x1 m1( ){ } E x2 m2( ) x2 m2( ){ }
E xn mn( ) x1 m1( ){ } E xn mn( ) xn mn( ){ }
Eigenvectors and Eigenvalues
EECS490: Digital Image Processing
Transform the data by a Hotelling* transformation
y = A x mx( )
A =
e1T
e2T
enT
Eigenvectors and Eigenvalues
where the rows of A are the eigenvectors of thecovariance matrix Cx
* Also known as the Karhunen-Loeve transformation
EECS490: Digital Image Processing
The transformed variable y has the properties that
my = E y{ } = 0
Cy = ACxAT=
1 0 0
0 2
0 n
1 > 2 > > n
Eigenvectors and Eigenvalues
Cy is diagonal which indicates that the transformed y vectors areuncorrelated.
where
and
EECS490: Digital Image Processing
x = AT y + mx
A =
e1T
e2T
ekT
Eigenvectors and Eigenvalues
This transformation can be invertedto give back the original imagevectors x
where
EECS490: Digital Image Processing
The difference (rms error) between this approximation and theoriginal x is given by the sum of the eigenvalues correspondingto the removed eigenvectors.
B =
e1T
e2T
ekT
x̂ = BT y + mx
erms = jj= k+1
n
Eigenvectors and Eigenvalues
What if I replace A by B which has only the keigenvectors corresponding to the k largesteigenvalues?
We now have an approximation to x given by
EECS490: Digital Image Processing
Eigenvectors and Eigenvalues
The transformed vectors y are orthogonal and form aorthoginal basis set for describing the original set of images.
Each yi is an eigenvector of Cx and represents a “feature” ofthe original set of images. y1 corresponds to the mostsignificant feature since y1 corresponds to the largesteigenvalue 1. y2 corresponds to the next most significantfeature since y2 corresponds to the next largest eigenvalue 2.
x̂ = ai yii=1
k
EECS490: Digital Image Processing
Eigenvector Example
© 2002 R. C. Gonzalez & R. E. Woods
Six registered multi-spectral images: blue, green, red, nearinfrared, middle infrared, and far (thermal) infrared bands.
EECS490: Digital Image Processing
Eigenvector Example
© 2002 R. C. Gonzalez & R. E. Woods
Each pixel in this multi-spectal image can be described by a six-component pixel vector x
These are 384x239 pixels so there are 91776 pixel vectorsdescribing the complete image
EECS490: Digital Image Processing
Eigenvector Example
© 2002 R. C. Gonzalez & R. E. Woods
We can use a computer program to compute the mean and 6x6covariance matrix Cx for these 91776 pixels. We can thencalculate the eigenvectors e1 to e6 for Cx. For this example theeigenvalues are
EECS490: Digital Image Processing
Eigenvector Example
© 2002 R. C. Gonzalez & R. E. Woods
This is the mostsignificant imagewith the mostinformation ( 1)
This is the leastsignificant imagewith the leastinformation ( 1)
These are the six principal component images given by yi=eiT(x-mx)
This transformation rearranges the information presented in the multi-spectral image.
y1 y2 y3
y4 y5 y6
EECS490: Digital Image Processing
Eigenvector Example
© 2002 R. C. Gonzalez & R. E. Woods
Now reconstruct the six original images using only the two mostsignificant eigenvectors.
y = A x mx( )
Compute
B =e1T
e2T
Use only
x̂ = BTy1y2
+ mx
Invert
EECS490: Digital Image Processing
Eigenvector Example
© 2002 R. C. Gonzalez & R. E. Woods
Original images
EECS490: Digital Image Processing
Eigenvector Example
© 2002 R. C. Gonzalez & R. E. Woods
Differencesbetween theoriginalimages andthe imagesreconstructedusing only twoeigenvectors.
EECS490: Digital Image Processing
Eigenface Analysis
© 2002 R. C. Gonzalez & R. E. Woods
Original dataset of registeredfaces
http://www.geop.ubc.ca/CDSST/eigenfaces.html.
Corresponding 15 eigenfaces
Eigenvector vector expansions have beenfinding increasing application in such areasas face recognition.
EECS490: Digital Image Processing
Eigenvalues & Eigenvectors
The first eigenvector e1 corresponds to the axis oflargest variance, i.e., the principal axis.
Translating the objectto its centroid (mean)and aligning the principalaxes (variances) canoften be used tofacilitate objectanalysis and recognition.
EECS490: Digital Image Processing
Eigenvector Example #2
© 2002 R. C. Gonzalez & R. E. Woods
1. Representthe boundarypoints of anobject as a setof 2-D vectorsx and computeCx
2. Compute the eigenvectors of Cx
3. Compute the mean mx andtranslate the mean to the origin
4. Rotate the objectso that e1 correspondsto the horizontal axisand e2 to the verticalaxis.
EECS490: Digital Image Processing
Manual Eigenvector Example
© 2002 R. C. Gonzalez & R. E. Woods
This is asimpleboundaryanalysiswhich you canreplicate byhand.
EECS490: Digital Image Processing
Relational Descriptors
© 2002 R. C. Gonzalez & R. E. Woods
This is similarto the texturegrammarswhich wediscussedearlier.
These are the re-writing rules whichcan be used to generate abab… fromS
a,b are the elements shown above; S is thestarting symbol; A is a variable
SaAabSabaAabab
etc.
EECS490: Digital Image Processing
Representation and Description
© 2002 R. C. Gonzalez & R. E. Woods
SaAabSabaAabab
The rules can berepeated to describelarger objects. Thenumbers correspond tothe rules which wereapplied.
EECS490: Digital Image Processing
Representation and Description
© 2002 R. C. Gonzalez & R. E. Woods
Object boundaries can be coded using head-to-tail connected directed line segments.
EECS490: Digital Image Processing
Representation and Description
© 2002 R. C. Gonzalez & R. E. Woods
Morecomplexrelationshipscan bedefined fordirected linesegments.
EECS490: Digital Image Processing
Representation and Description
© 2002 R. C. Gonzalez & R. E. Woods
EECS490: Digital Image Processing
Representation and Description
© 2002 R. C. Gonzalez & R. E. Woods
EECS490: Digital Image Processing
Object Recognition
This shows a clearly defined region for Irissetosa based upon petal length and width.
© 2002 R. C. Gonzalez & R. E. Woods
EECS490: Digital Image Processing
Object Recognition
Signatures can be used to code objects to berecognized.
© 2002 R. C. Gonzalez & R. E. Woods
EECS490: Digital Image Processing
Object Recognition
String descriptions can also be used to code objects.
© 2002 R. C. Gonzalez & R. E. Woods
EECS490: Digital Image Processing
Object Recognition
More complexstructures suchas trees areneeded todescribe theobjects in atypical image
© 2002 R. C. Gonzalez & R. E. Woods
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Minimum Distance Classifiers
The prototype for each class (out of W classes) is the meanvector of that class
mj =1
N j
x jx wj
j = 1,2,...,W
Compute “closeness” using Euclidian distance
Dj x( ) = x mj j = 1,2,...,W
Assign x to class j if Dj(x) is the smallest distance
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Minimum Distance Classifiers
More mathematically the distance from a candidate pattern xto the pairs of mean vectors mi and mj can be written as
This describes a plane marking the classification boundarybetween classes i and j, i.e.,
If dij(x) >0 then assign x to class j otherwise assign to class i
dij x( ) = di x( ) dj x( ) = xT mi mj( )1
2mi mj( )
Tmi + mj( )
dij x( ) = 0
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
d12 x( ) = 0
Decision classifier
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
Check characters areread horizontally by asingle vertical sensorwhich travels from leftto right to generate adistinct waveform foreach character
The characters andsensors are optimized togive distinct responses ina simple x-y grid.
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
Correlation can be used tofind similar patterns
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Statistical Pattern Recognition
If the class probabilities are not equal we must use a Bayesianclassifier and use the maximum to classify unknown patterns
The threshold for classifying patterns is where thepdf’s overlap
dj x( ) = p x | j( )P j( ) j = 1,2,...,W
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
In moredimensions theclassifier forms aplane (orhyperplane) forGaussian patternclasses
dj x( ) = lnP j( )1
2ln Cj
1
2x mj( )
TCj
1 x mj( ) j = 1,...,W
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
We can use the same registered multi-spectralimage data as input to a pattern classifier
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
Prototypes
(3)Vegetationclassification
(1) Waterclassification
(2) Urban dev.classification
Errors usinghalf of theprototypes totrain, theother toclassify
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
Bayes classification of pixels inthe training regions 1,2,3
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
The perceptron is an early neuralnetwork architecture forlearning decision classifiers
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
Neural networks can be used toimplement (and learn)geometrically complex decisionfunctions.
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Object Recognition
Multi-layer networks can be used toapproximate (learn) more complexdecision functions.
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Shape Number Classification
Objects can besimilar up todifferent ordersof shape number.This can be usedto classifyobjects.
EECS490: Digital Image Processing
© 2002 R. C. Gonzalez & R. E. Woods
Shape Number Classification