this is a shortened version of the tutorial given at the...
TRANSCRIPT
![Page 1: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/1.jpg)
Aleš Leonardis
Faculty of Computer and Information Science
University of Ljubljana
Slovenia
Subspace Methods for Visual Learning and Recognition
This is a shortened version of the tutorial given at the ECCV’2002, Copenhagen, and ICPR’2002, Quebec City.
© Copyright 2002 by Aleš Leonardis, University of Ljubljana, and Horst Bischof, Graz University of Technology
![Page 2: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/2.jpg)
2Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Outline Part 1
♦ Motivation
♦ Appearance based learning and recognition
♦ Subspace methods for visual object recognition
♦ Principal Components Analysis (PCA)
♦ Linear Discriminant Analysis (LDA)
♦ Canonical Correlation Analysis (CCA)
♦ Independent Component Analysis (ICA)
♦ Non-negative Matrix Factorization (NMF)
♦ Kernel methods for non-linear subspaces
![Page 3: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/3.jpg)
3Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Outline Part 2
♦ Robot localization
♦ Robust representations and recognition
♦ Robust PCA recognition
♦ Scale invariant recognition using PCA
♦ Illumination insensitive recognition
♦ Representations for panoramic images
♦ Incremental building of eigenspaces
♦ Multiple eigenspaces for efficient representation
♦ Robust building of eigenspaces
♦ Research issues
![Page 4: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/4.jpg)
4Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Learning and recognition
scene trainingimages
input image
3D reconstruction
learning
matching
matching
![Page 5: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/5.jpg)
5Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Appearance-based approaches
Attention in the appearance-based approaches
Encompass combined effects of:
• shape,
• reflectance properties,
• pose in the scene,
• illumination conditions.
Acquired through an automatic learning phase.
![Page 6: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/6.jpg)
6Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Appearance-based approaches
Objects are represented by a large number of views:
![Page 7: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/7.jpg)
7Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Subspace Methods
• Images are represented as points in the N-dimensional vector space• Set of images populate only a small fraction of the space• Characterize subspace spanned by images
… …
…Image set Basis images Representation
≈
![Page 8: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/8.jpg)
8Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Subspace Methods
Properties of the representation:
• Optimal Reconstruction ⇒ PCA
• Optimal Separation ⇒ LDA
• Optimal Correlation ⇒ CCA
• Independent Factors ⇒ ICA
• Non-negative Factors ⇒ NMF
• Non-linear Extension ⇒ Kernel Methods
![Page 9: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/9.jpg)
9Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Eigenspace representation
♦ Image set (normalised, zero-mean)
♦ We are looking for orthonormal basis functions:
♦ Individual image is a linear combination of basis functions
22
1
2
11
2
||)()(||||))()((||
||)()(||||||
yxuyx
uyuxyx
jj
k
jjjj
k
jjj
k
jjj
qqqq
−=−
=−≈−
∑
∑∑
=
==
![Page 10: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/10.jpg)
10Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Best basis functions ν?
♦ Optimisation problem
♦ Taking the k eigenvectors with the largest eigenvalues of
♦ PCA or Karhunen-Loéve Transform (KLT)
![Page 11: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/11.jpg)
11Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Efficient eigenspace computation
♦ n << m
♦ Compute the eigenvectors u'i, i = 0,...,n-1, of the inner product matrix
♦ The eigenvectors of XXT can be obtained by using XXTXvi'=λ 'iXvi':
![Page 12: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/12.jpg)
12Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Principal Component Analysis
![Page 13: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/13.jpg)
13Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Principal Component Analysis
= q1⋅ + q2⋅ + q3⋅ + ...
![Page 14: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/14.jpg)
14Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Image representation with PCA
u1
u2
u3
![Page 15: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/15.jpg)
15Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Image presentation with PCA
![Page 16: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/16.jpg)
Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Properties PCA
♦ It can be shown that the mean square error between xi and its reconstruction using only m principle eigenvectors is given by the expression :
♦ PCA minimizes reconstruction error
♦ PCA maximizes variance of projection
♦ Finds a more “natural” coordinate system for the sample data.
∑∑∑+===
=−N
mjj
m
jj
N
jj
111
λλλ
![Page 17: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/17.jpg)
17Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
PCA for visual recognition and pose estimation
Objects are represented as coordinates in an n-dimensional eigenspace.
An example:
3-D space with points representing individual objects or a manifold representing parametric eigenspace (e.g., orientation, pose, illumination).
u0 u2
u1
![Page 18: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/18.jpg)
18Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
PCA for visual recognition and pose estimation
♦ Calculate coefficients
♦ Search for the nearest point (individual or on the curve)
♦ Point determines object and/or pose
![Page 19: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/19.jpg)
19Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Calculation of coefficients
To recover qi the image is projected onto the eigenspace
• Complete image x is required to calculate qi.
• Corresponds to Least-Squares Solution
∑−
=
≤≤>==<1
1
1)(n
jijji kiuxq iux,x
< > = q1< > + q2< > + ... =q1
< > = q1< > + q2< > + ... =q2
![Page 20: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/20.jpg)
20Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Linear Discriminant Analysis (LDA)
♦ PCA minimizes projection error
PCA-Projection
Best discriminatingProjection
♦ PCA is „unsupervised“ no information on classes is used
♦ Discriminating information might be lost
![Page 21: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/21.jpg)
21Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
LDA
♦ Linear Discriminance Analysis (LDA)
– Maximize distance between classes – Minimize distance within a class
Fisher Linear Discriminance
wSwwSw
wW
BT
T
=)(ρ
![Page 22: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/22.jpg)
22Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
LDA: Problem formulation
♦ n Sample images:
♦ c classes:
♦ Average of each class:
♦ Total average:
{ }nxx ,,1 Λ
{ }cχχ ,,1 Λ
∑∈
=ikx
ki
i xn χ
µ1
∑=
=N
kkx
n 1
1µ
![Page 23: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/23.jpg)
23Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
LDA: Practice
♦ Scatter of class i: ( )( )Tik
xiki xxS
ik
µµχ
−∑ −=∈
∑==
c
iiW SS
1
( )( )∑ −−==
c
i
TiiiBS
1µµµµχ
BWT SSS +=
♦ Within class scatter:
♦ Between class scatter:
♦ Total scatter:
![Page 24: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/24.jpg)
24Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Good separation
2S
1S
BS
21 SSSW +=
LDA
![Page 25: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/25.jpg)
25Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
LDA
♦ Maximization of
♦ is given by solution of generalized eigenvalue problem
♦ For the c-class case we obtain (at most) c-1 projections as the largest eigenvalues of
wSwwSw
wW
BT
T
=)(ρ
wSwS wB λ=
ii wSwS wB λ=
![Page 26: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/26.jpg)
26Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
LDA
♦ Example Fisherface of recognition Glasses/NoGlasses(Belhumeur et.al. 1997)
![Page 27: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/27.jpg)
27Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Canonical Correlation Analysis (CCA)
♦ Also „supervised“ method but motivated by regression tasks, e.g. pose estimation.
♦ Canonical Correlation Analysis relates two sets of observations by determining pairs of directions that yield maximum correlation between these sets.
♦ Find a pair of directions (canonical factors) wx∈ ℜp, wy∈ ℜq, so that the correlation of the projections c = wx
Tx and d = wyTy
becomes maximal.
![Page 28: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/28.jpg)
28Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
What is CCA?
yyyTyxxx
Tx
yxyTx
yTT
yxTT
x
yTT
x
EE
E
dEcE
cdE
wCwwCw
wCw
wyywwxxw
wxyw=
==
][][
][
][][
][22
ρCanonicalCorrelation0 ≤ r ≤ 1
Between SetCovariance
Matrix
![Page 29: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/29.jpg)
29Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
What is CCA?
=
=
yy
xx
yx
xy
CC
BC
CA
00
,0
0
• Finding solutions
=
y
x
ww
w
BwwAww
T
T
r =
Rayleigh Quotient
BwAw µ=
Generalized Eigenproblem
![Page 30: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/30.jpg)
31Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
CCA Example
0.4 0.3 0.2 0.1 0 0.1 0.2 0.3
0.30.2
0.10
0.10.2
0.3
0.3
0.2
0.1
0
0.1
0.2
0.3
Parametric eigenspace obtained by PCA for 2DoF in pose
![Page 31: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/31.jpg)
32Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
CCA Example
5 4 3 2 1 0 1 2 3 4 50.01
0.008
0.006
0.004
0.002
0
0.002
0.004
0.006
0.008
0.01
CCA representation(projections of training images onto wx1, wx2)
![Page 32: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/32.jpg)
33Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Independent Component Analysis (ICA)
♦ ICA is a powerful technique from signal processing (Blind Source Separation)
♦ Can be seen as an extension of PCA
♦ PCA takes into account only statistics up to 2nd order
♦ ICA finds components that are statistically independent (or as independent as possible)
![Page 33: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/33.jpg)
34Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Independent Component Analysis (ICA)
♦ m scalar variables X=(x1 ... xm)T
♦ They are assumed to be obtained as linear mixtures of n sources S=(s1 ... sn)T
♦ Task: Given X find A, S (under the assumption that S are independent)
ASX =
![Page 34: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/34.jpg)
35Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
ICA Example
Original Sources
Mixtures
Recovered Sources
![Page 35: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/35.jpg)
36Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
ICA Example
ICA basis obtainedfrom 16x16 patchesof natural images (Bell&Sejnowski 96)
![Page 36: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/36.jpg)
37Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Face Recognition using ICA
♦ PCA vs. ICA on Ferret DB (Baek et.al. 02)
PCA
ICA
![Page 37: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/37.jpg)
38Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Non-Negative Matrix Factorization (NMF)
♦ How can we obtain part-based representation?
♦ Local representation where parts are added
♦ E.g. learn from a set of faces the parts a face consists of, i.e. eyes, nose, mouth, etc.
♦ Non-Negative Matrix Factorization (Lee & Seung 1999) lead to part based representation
![Page 38: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/38.jpg)
39Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Matrix Factorization - Constraints
V ≈ WH♦ PCA: W are orthonormal basis vectors
♦ VQ : H are unity vectors
♦ NMF: V,W,H are non-negative
ijjin wwwwwW δ=⋅= ],,,,[ 21 Λ
]0,,0,1,0,0[ ],,,,[ 21 ΛΛ == Tjn hhhhH
jiHWV ijijij , 0,, ∀≥
![Page 39: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/39.jpg)
41Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Learning
Training images Basis images
Find basis images from the training set
![Page 40: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/40.jpg)
42Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Face features
Basis images
Encoding (Coefficients)
Reconstructed image
![Page 41: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/41.jpg)
43Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Kernel Methods
♦ All presented methods are linear
♦ Can we generalize to non-linear methods in a computational efficient manner?
![Page 42: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/42.jpg)
44Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Kernel Methods
♦ Kernel Methods are powerful methods (introduced with Support Vector Machines) to generalize linear methods
BASIC IDEA:
1. Non-linear mapping of data in high dimensional space
2. Perform linear method in high-dimensional space
Non-linear method in original space
![Page 43: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/43.jpg)
45Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Outline Part 2
♦ Robot localization
♦ Robust representations and recognition
♦ Robust recognition using PCA
♦ Scale invariant recognition using PCA
♦ Illumination insensitive recognition
♦ Representations for panoramic images
♦ Incremental building of eigenspaces
♦ Multiple eigenspaces for efficient representation
♦ Robust building of eigenspaces
♦ Research issues
![Page 44: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/44.jpg)
46Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Appearance-based approaches
A variety of successful applications:
• Human face recognition e.g. [Turk & Pentland]
• Visual inspection e.g. [Yoshimura & Kanade]
• Visual positioning and tracking of robot manipulators, e.g. [Nayar & Murase]
• Tracking e.g., [Black & Jepson]
• Illumination planning e.g., [Murase & Nayar]
• Image spotting e.g., [Murase & Nayar]
• Mobile robot localization e.g., [Jogan & Leonardis]
• Background modeling e.g., [Oliver, Rosario & Pentland]
![Page 45: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/45.jpg)
47Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Mobile Robot
![Page 46: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/46.jpg)
48Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Panoramic image
![Page 47: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/47.jpg)
49Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Environment map
♦environments are represented by a large number of views
♦localisation = recognition
![Page 48: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/48.jpg)
50Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Compression with PCA
![Page 49: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/49.jpg)
51Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Image representation with PCA
![Page 50: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/50.jpg)
52Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Localisation
![Page 51: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/51.jpg)
53Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Distance vs. similarity
![Page 52: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/52.jpg)
54Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Robot localisation
♦ Interpolated hyper-surface represents the memorized environment.
♦ The parameters to be retrieved are related to position and orientation.
♦ Parameters of an input image are obtained by scalar product.
![Page 53: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/53.jpg)
55Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Localisation
![Page 54: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/54.jpg)
56Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Enhancing recognition and representations
♦ Occlusions, varying background, outliers – Robust recognition using PCA
♦ Scale variance– Multiresolution coefficient estimation– Scale invariant recognition using PCA
♦ Illumination variations– Illumination insensitive recognition
♦ Rotated panoramic images– Spinning eigenimages
♦ Incremental building of eigenspaces
♦ Multiple eigenspaces for efficient representations
♦ Robust building of eigenspaces
![Page 55: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/55.jpg)
57Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Occlusions
![Page 56: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/56.jpg)
58Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Calculation of coefficients
To recover qi the image is projected onto the eigenspace
• Complete image x is required to calculate qi.
• Corresponds to Least-Squares Solution
∑−
=
≤≤>==<1
1
1)(n
jijji kiuxq iux,x
< > = q1< > + q2< > + ... =q1
< > = q1< > + q2< > + ... =q2
![Page 57: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/57.jpg)
59Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Non-robustness
![Page 58: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/58.jpg)
60Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Robust method
![Page 59: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/59.jpg)
61Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Robust algorithm
![Page 60: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/60.jpg)
62Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Selection
![Page 61: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/61.jpg)
63Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Robust recovery of coefficients
![Page 62: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/62.jpg)
66Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Robust localisation under occlusions
![Page 63: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/63.jpg)
67Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Robust localisation at 60% occlusion
Standard approach Robust approach
![Page 64: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/64.jpg)
68Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Mean error of localisation
♦ Mean error of localisation with respect to % of occlusion
![Page 65: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/65.jpg)
69Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Illumination insensitive recognition
• Recognition of objects under
varying illumination
• global illumination changes
• highlights
• shadows
• Dramatic effects of illumination on
objects appearance
• Training set under a single
ambient illumination
![Page 66: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/66.jpg)
70Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Illumination insensitive recognition
Our Approach
• Global eigenspace representation
• Local gradient based filters
• Efficient combination of global and local representations
• Robust coefficient recovery in eigenspaces
![Page 67: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/67.jpg)
71Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Eigenspaces and filtering
![Page 68: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/68.jpg)
72Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Filtered eigenspaces
![Page 69: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/69.jpg)
73Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Gradient-based filters
Global illuminationGlobal illumination
Gradient-based filtersGradient-based filters
Steerable filters [Simoncelli]
![Page 70: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/70.jpg)
74Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Robust coefficient recovery
Highlights and shadowsHighlights and shadows
Robust coefficient recoveryRobust coefficient recovery
Λ+++=321
aaa
Λ+++=321
aaa
Λ+++=321
aaaΜ
Λ+++= 321 aaaΜ
Hypothesize &
Select
Hypothesize &
Select
Robust solution of linear equations
![Page 71: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/71.jpg)
75Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Experimental results
Test images Standard methodOur approach
à Demo
![Page 72: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/72.jpg)
76Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Experimental results
obj. 1 2 3 4 5 % ang.1 360 0 0 0 0 100.0 5.252 0 308 16 0 0 95.1 10.553 0 0 504 0 0 100.0 1.054 19 4 3 332 2 92.2 3.375 15 2 17 0 578 94.4 3.34avg. 96.4 4.19
Robust filtered method - all eigenvectors used
Standard method - all eigenvectors used
obj. 1 2 3 4 5 % ang.1 141 0 14 26 179 39.2 10.502 0 254 62 5 3 78.4 18.903 0 4 317 0 183 62.9 3.474 23 6 38 249 44 69.2 7.115 3 1 51 0 557 91.0 6.82avg. 70.3 8.53
![Page 73: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/73.jpg)
77Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Research issues
♦ Comparative studies (e.g., LDA versus PCA, PCA versus ICA)
♦ Robust learning of other representations (e.g. LDA, CCA)
♦ Integration of robust learning with modular eigenspaces
♦ Local versus Global subspace represenations
♦ Combination of subspace representations in a hierarchical framework
![Page 74: This is a shortened version of the tutorial given at the ...hic/8803-Fall-09/slides/SubSpace-Learning.pdfSubspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL 38](https://reader031.vdocument.in/reader031/viewer/2022011905/5f2c04595b97c377db7f7c04/html5/thumbnails/74.jpg)
78Subspace Methods for Visual Learning and Recognition Aleš Leonardis, UOL
Further readings
♦ Recognizing objects by their appearance using eigenimages (SOFSEM 2000, LNCS 1963)
♦ Robust recognition using eigenimages (CVIU 2000, Special Issue on Robust Methods in CV)
♦ Illumination insensitive eigenspaces (ICCV 2001)
♦ Mobile robot localization under varying illumination (ICPR 2002)
♦ Eigenspace of spinning images (OMNI 2000, ICPR 2000, ICAR 2001)
♦ Incremental building of eigenspaces (ICRA 2002, ICPR 2002, CogVis 2002)
♦ Multiple eigenspaces (Pattern Recognition 2002)
♦ Robust building of eigenspaces (ECCV 2002)
♦ Special issue of Pattern Recognition on Kernel and Subspace Methods in Computer Vision (Guest Editors A. Leonardis and H. Bischof), to appear in 2003.