lecture face recognitionface recognition • digital photography • surveillance • album...
TRANSCRIPT
![Page 1: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/1.jpg)
COS 429: COMPUTER VISONFace Recognition
• Intro to recognition• PCA and Eigenfaces• LDA and Fisherfaces• Face detection: Viola & Jones• (Optional) generic object models for faces: the Constellation Model
Reading: Turk & Pentland, ???
![Page 2: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/2.jpg)
Face Recognition• Digital photography
![Page 3: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/3.jpg)
Face Recognition• Digital photography• Surveillance
![Page 4: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/4.jpg)
Face Recognition• Digital photography• Surveillance• Album organization
![Page 5: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/5.jpg)
Face Recognition• Digital photography• Surveillance• Album organization• Person tracking/id.
![Page 6: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/6.jpg)
Face Recognition• Digital photography• Surveillance• Album organization• Person tracking/id.• Emotions and
expressions
![Page 7: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/7.jpg)
Face Recognition• Digital photography• Surveillance• Album organization• Person tracking/id.• Emotions and
expressions• Security/warfare• Tele-conferencing• Etc.
![Page 8: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/8.jpg)
vs.
Identification orDiscrimination
What’s ‘recognition’?
![Page 9: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/9.jpg)
What’s ‘recognition’?
vs.
Categorization or Classification
Identification orDiscrimination
![Page 10: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/10.jpg)
What’s ‘recognition’?
Categorization or Classification
Identification orDiscrimination
No
loca
lizat
ion
Yes, there are faces
![Page 11: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/11.jpg)
What’s ‘recognition’?
Categorization or Classification
Identification orDiscrimination
No
loca
lizat
ion
Yes, there is John Lennon
![Page 12: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/12.jpg)
What’s ‘recognition’?
Categorization or Classification
Identification orDiscrimination
No
loca
lizat
ion
Det
ectio
n or
Lo
caliz
atoi
n
John Lennon
![Page 13: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/13.jpg)
What’s ‘recognition’?
Categorization or Classification
Identification orDiscrimination
No
loca
lizat
ion
Det
ectio
n or
Lo
caliz
atoi
n
![Page 14: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/14.jpg)
What’s ‘recognition’?
Categorization or Classification
Identification orDiscrimination
No
loca
lizat
ion
Det
ectio
n or
Lo
caliz
atoi
n
![Page 15: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/15.jpg)
Today’s agenda
Categorization or Classification
Identification orDiscrimination
No
loca
lizat
ion
Det
ectio
n or
Lo
caliz
atoi
n
1. PCA & Eigenfaces2. LDA & Fisherfaces
3. AdaBoost
4. Constellation model
![Page 16: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/16.jpg)
![Page 17: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/17.jpg)
Eigenfaces and Fishfaces
• Introduction• Techniques
– Principle Component Analysis (PCA)– Linear Discriminant Analysis (LDA)
• Experiments
![Page 18: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/18.jpg)
The Space of Faces
• An image is a point in a high dimensional space– An N x M image is a point in RNM
– We can define vectors in this space as we did in the 2D case
+=
[Thanks to Chuck Dyer, Steve Seitz, Nishino]
![Page 19: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/19.jpg)
Key Idea
}ˆ{ PRLx=χ• Images in the possible set are highly correlated.
• So, compress them to a low-dimensional subspace thatcaptures key appearance characteristics of the visual DOFs.
• EIGENFACES: [Turk and Pentland]
USE PCA!
![Page 20: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/20.jpg)
Two simple but useful techniquesFor example, a generative graphical model:P(identity,image) = P(identiy|image) P(image)
Preprocessing model(can be performed by PCA)
![Page 21: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/21.jpg)
Principal Component Analysis (PCA) Principal Component Analysis (PCA)
• PCA is used to determine the most representing features among data points. – It computes the p-dimensional subspace such that the
projection of the data points onto the subspace has the largest variance among all p-dimensional subspaces.
![Page 22: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/22.jpg)
Illustration of PCAIllustration of PCA
One projection PCA projection
x1
x2
1 2
3
4
5
6
x1
x2
1 2
3
4
5
6
X1’
![Page 23: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/23.jpg)
Illustration of PCAIllustration of PCA
x1
x2
1st principal component
2rd principal component
![Page 24: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/24.jpg)
EigenfaceEigenface for Face Recognitionfor Face Recognition• PCA has been used for face image
representation/compression, face recognition and many others.
• Compare two faces by projecting the images into the subspace and measuring the EUCLIDEAN distance between them.
![Page 25: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/25.jpg)
Mathematical FormulationFind a transformation, W,
m-dimensional n-dimensionalOrthonormal
Total scatter matrix:
Wopt corresponds to m eigen-vectors of ST
![Page 26: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/26.jpg)
Eigenfaces• PCA extracts the eigenvectors of A
– Gives a set of vectors v1, v2, v3, ...– Each one of these vectors is a direction in face space
• what do these look like?
![Page 27: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/27.jpg)
Projecting onto the Eigenfaces
• The eigenfaces v1, ..., vK span the space of faces
– A face is converted to eigenface coordinates by
![Page 28: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/28.jpg)
AlgorithmAlgorithm
1. Align training images x1, x2, …, xN
2. Compute average face u = 1/N Σ xi
3. Compute the difference image φi = xi – u
TrainingTraining
Note that each image is formulated into a long vector!
![Page 29: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/29.jpg)
AlgorithmAlgorithm
Testing
1. Projection in EigenfaceProjection ωi = W (X – u), W = {eigenfaces}
2. Compare projections
ST = 1/NΣφi φiT = BBT, B=[φ1, φ2 … φN]
4. Compute the covariance matrix (total scatter matrix)
5. Compute the eigenvectors of the covariance matrix , W
![Page 30: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/30.jpg)
Illustration of Illustration of EigenfacesEigenfaces
These are the first 4 eigenvectors from a training set of 400 images (ORL Face Database). They look like faces, hence called Eigenface.
The visualization of eigenvectors:
![Page 31: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/31.jpg)
Eigenfaces look somewhat like generic faces.
![Page 32: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/32.jpg)
EigenvaluesEigenvalues
![Page 33: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/33.jpg)
Only selecting the top P eigenfaces reduces the dimensionality.Fewer eigenfaces result in more information loss, and hence less discrimination between faces.
Reconstruction and ErrorsReconstruction and Errors
P = 4
P = 200
P = 400
![Page 34: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/34.jpg)
Summary for PCA and Summary for PCA and EigenfaceEigenface• Non-iterative, globally optimal solution• PCA projection is optimal for reconstruction
from a low dimensional basis, but may NOT be optimal for discrimination…
![Page 35: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/35.jpg)
Linear Discriminant Analysis (LDA)Linear Discriminant Analysis (LDA)• Using Linear Discriminant Analysis (LDA) or
Fisher’s Linear Discriminant (FLD) • Eigenfaces attempt to maximise the scatter of the
training images in face space, while Fisherfacesattempt to maximise the between class scatter, while minimising the within class scatter.
![Page 36: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/36.jpg)
Illustration of the ProjectionIllustration of the Projection
Poor Projection Good Projection
x1
x2
x1
x2
Using two classes as example:
![Page 37: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/37.jpg)
Comparing with PCAComparing with PCA
![Page 38: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/38.jpg)
Variables
• N Sample images: • c classes:
• Average of each class:
• Total average:
{ }Nxx ,,1 L
{ }cχχ ,,1 L
∑=∈ ikx
ki
i xN χ
μ 1
∑==
N
kkx
N 1
1μ
![Page 39: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/39.jpg)
Scatters
• Scatter of class i: ( )( )Tikx
iki xxSik
μμχ
−∑ −=∈
∑==
c
iiW SS
1
( )( )∑ −−==
c
i
TiiiBS
1μμμμχ
BWT SSS +=
• Within class scatter:
• Between class scatter:
• Total scatter:
![Page 40: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/40.jpg)
Illustration
2S
1S
BS
21 SSSW +=
x1
x2
![Page 41: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/41.jpg)
Mathematical Formulation (1)
After projection:
Between class scatter (of y’s):Within class scatter (of y’s):
kT
k xWy =
WSWS BT
B =~
WSWS WT
W =~
![Page 42: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/42.jpg)
Mathematical Formulation (2)• The desired projection:
WSW
WSW
SS
WW
TB
T
W
Bopt WW
max arg~~
max arg ==
miwSwS iWiiB ,,1 K== λ• How is it found ? Generalized Eigenvectors
Data dimension is much larger than the number of samplesThe matrix is singular:
Nn >>
( ) cNSRank W −≤WS
![Page 43: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/43.jpg)
Fisherface (PCA+FLD)
• Project with FLD to space
• Project with PCA to space
1−c kT
fldk zWy =
WWSWW
WWSWWW
pcaWTpca
TpcaB
Tpca
T
fld Wmax arg=
cN − kT
pcak xWz =
WSWW TT
pca Wmax arg=
![Page 44: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/44.jpg)
Illustration of FisherFace• Fisherface
![Page 45: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/45.jpg)
Results: Eigenface vs. Fisherface (1)
• Variation in Facial Expression, Eyewear, and Lighting
• Input: 160 images of 16 people• Train: 159 images• Test: 1 image
With glasses
Without glasses
3 Lighting conditions
5 expressions
![Page 46: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/46.jpg)
Eigenface vs. Fisherface (2)
![Page 47: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/47.jpg)
discussion• Removing the first three principal
components results in better performance under variable lighting conditions
• The Firsherface methods had error rates lower than the Eigenface method for the small datasets tested.
![Page 48: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/48.jpg)
Today’s agenda
Categorization or Classification
Identification orDiscrimination
No
loca
lizat
ion
Det
ectio
n or
Lo
caliz
atoi
n
1. PCA & Eigenfaces2. LDA & Fisherfaces
3. AdaBoost
4. Constellation model
![Page 49: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/49.jpg)
Robust Face Detection Using AdaBoost
• Brief intro on (Ada)Boosting• Viola & Jones, 2001
– Weak detectors: Haar wavelets– Integral image– Cascade– Exp. & Res.
Reference:P. Viola and M. Jones (2001) Robust Real-time Object Detection, IJCV.
![Page 50: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/50.jpg)
Discriminative methodsObject detection and recognition is formulated as a classification problem.
Bag of image patches
Decision boundary
… and a decision is taken at each window about if it contains a target object or not.
Computer screen
Background
In some feature space
Where are the screens?
The image is partitioned into a set of overlapping windows
![Page 51: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/51.jpg)
A simple object detector with Boosting Download
• Toolbox for manipulating dataset
• Code and dataset
Matlab code
• Gentle boosting
• Object detector using a part based model
Dataset with cars and computer monitors
http://people.csail.mit.edu/torralba/iccv2005/
![Page 52: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/52.jpg)
• A simple algorithm for learning robust classifiers– Freund & Shapire, 1995– Friedman, Hastie, Tibshhirani, 1998
• Provides efficient algorithm for sparse visual feature selection– Tieu & Viola, 2000– Viola & Jones, 2003
• Easy to implement, not requires external optimization tools.
Why boosting?
![Page 53: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/53.jpg)
• Defines a classifier using an additive model:
Boosting
Strong classifier
Weak classifier
WeightFeaturesvector
![Page 54: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/54.jpg)
• Defines a classifier using an additive model:
• We need to define a family of weak classifiers
Boosting
Strong classifier
Weak classifier
WeightFeaturesvector
from a family of weak classifiers
![Page 55: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/55.jpg)
Each data point has
a class label:
wt =1and a weight:
+1 ( )
-1 ( )yt =
Boosting• It is a sequential procedure:
xt=1
xt=2
xt
![Page 56: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/56.jpg)
Toy exampleWeak learners from the family of lines
h => p(error) = 0.5 it is at chance
Each data point has
a class label:
wt =1and a weight:
+1 ( )
-1 ( )yt =
![Page 57: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/57.jpg)
Toy example
This one seems to be the best
Each data point has
a class label:
wt =1and a weight:
+1 ( )
-1 ( )yt =
This is a ‘weak classifier’: It performs slightly better than chance.
![Page 58: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/58.jpg)
Toy example
We set a new problem for which the previous weak classifier performs at chance again
Each data point has
a class label:
wt wt exp{-yt Ht}
We update the weights:
+1 ( )
-1 ( )yt =
![Page 59: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/59.jpg)
Toy example
We set a new problem for which the previous weak classifier performs at chance again
Each data point has
a class label:
wt wt exp{-yt Ht}
We update the weights:
+1 ( )
-1 ( )yt =
![Page 60: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/60.jpg)
Toy example
We set a new problem for which the previous weak classifier performs at chance again
Each data point has
a class label:
wt wt exp{-yt Ht}
We update the weights:
+1 ( )
-1 ( )yt =
![Page 61: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/61.jpg)
Toy example
We set a new problem for which the previous weak classifier performs at chance again
Each data point has
a class label:
wt wt exp{-yt Ht}
We update the weights:
+1 ( )
-1 ( )yt =
![Page 62: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/62.jpg)
Toy example
The strong (non- linear) classifier is built as the combination of all the weak (linear) classifiers.
f1 f2
f3
f4
![Page 63: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/63.jpg)
Real-time Face Detection• Integral Image
– New image representation to compute the features very quickly
• AdaBoost– Selecting a small number of important feature
• Cascade– A method for combining classifiers – Focussing attention on promising regions of the
image• Implemented on 700MHz Intel Pentium Ⅲ, face
detection proceeds at 15f/s.– Working only with a single grey scale image
![Page 64: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/64.jpg)
Features
• Three kinds of rectangle features
• The sum of the pixels which lie within the white rectangles are subtracted from the sum of pixels in the gray rectangles
![Page 65: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/65.jpg)
Integral Image
The sum within D=4-(2+3)+1
,
( , ) ( , )x x y y
ii x y i x y′ ′≤ ≤
′ ′= ∑
![Page 66: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/66.jpg)
Learning Classification Function (1)
…..
1( ,1)x 2( ,1)x 3( ,0)x 4( ,0)x
( , )n nx y①
② Initialize weights1,
1 1, for 1,0 respectively2 2i iw yl m
= =
# of face # of nonface
• Selecting a small number of important features
1α
2α
5( ,0)x 6( ,0)x
![Page 67: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/67.jpg)
Learning Classification Function (2)③ For t=1,….,T
,,
,1
t it i n
t jj
ww
w=
←∑
a. Normalize the weights
b. For each feature, j ( )j i j i iiw h x yε = −∑
1 if ( )( )
0 otherwisej j
j
f xh x
θ>⎧= ⎨⎩
c. Choose the classifier, ht with the lowest error tε
d. Update the weights 1 ( )1, ,
t i ih x yt i t i tw w β − −+ =
1t
tt
εβε
=−
1 or 0
![Page 68: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/68.jpg)
Learning Classification Function (3)④ The final strong classifier is
1 1
11 ( )( ) 2
0 otherwise
T Tt t tt th x
h xα α
= =
⎧ ≥⎪= ⎨⎪⎩
∑ ∑1logt
t
αβ
=
☞The final hypothesis is a weighted linear combination of the T hypotheses where the weights are inversely proportional to thetraining errors
1α
2α
![Page 69: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/69.jpg)
Requiring 0.7 seconds to scan 384x288 pixel image
Learning Results• 200 features• Detection rate: 95% - false positive 1/14,804
The first and second features selected by
AdaBoost
![Page 70: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/70.jpg)
The Attentional Cascade
• Reject many of the negative sub-windows
Two-feature strong classifier
Detect: 100%False positive:40%
![Page 71: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/71.jpg)
A Cascaded Detector
1 2 3
T T T
F F F
f, d
Ftarget
f, d f, d
![Page 72: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/72.jpg)
Detector Cascade Discussion
☞ The speed of the cascaded classifier is almost 10 times faster
![Page 73: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/73.jpg)
Experimental Results (1)• Training dataset
– Face training set: 4916 hand labeled faces– Scaled and aligned to a base resolution of 24ⅹ24
• Structure of the detector cascade– 32layer, 4297 feature
• Training time for the entire 32 layer detector – on the order of weeks – a single 466 MHz AlphaStation XP900
1 3 2T T T
F F F
2 20 50
5T
F
100
20T
F
200
Reject about 60% of non-facesCorrectly detecting close to 100% of faces
![Page 74: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/74.jpg)
Face Image Databases
• Databases for face recognition can be best utilized as training sets– Each image consists of an individual on a
uniform and uncluttered background• Test Sets for face detection
– MIT, CMU (frontal, profile), Kodak
![Page 75: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/75.jpg)
Training dataset: 4916 images
![Page 76: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/76.jpg)
Experimental Results
• Test dataset– MIT+CMU frontal face test set– 130 images with 507 labeled frontal faces
MIT test set: 23 images with 149 facesSung & poggio: detection rate 79.9% with 5 false positiveAdaBoost: detection rate 77.8% with 5 false positives
89.990.1-89.2---86.083.2Neural-net
93.791.891.190.890.189.888.885.278.3AdaBoost
422167110957865503110False detection
-> not significantly different accuracy
-> but the cascade class. almost 10 times faster
![Page 77: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/77.jpg)
![Page 78: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/78.jpg)
![Page 79: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/79.jpg)
![Page 80: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/80.jpg)
![Page 81: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/81.jpg)
![Page 82: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/82.jpg)
![Page 83: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/83.jpg)
![Page 84: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/84.jpg)
![Page 85: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/85.jpg)
Today’s agenda
Categorization or Classification
Identification orDiscrimination
No
loca
lizat
ion
Det
ectio
n or
Lo
caliz
atoi
n
1. PCA & Eigenfaces2. LDA & Fisherfaces
3. AdaBoost
4. Constellation model
![Page 86: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/86.jpg)
• Fischler & Elschlager 1973
• Yuille ‘91• Brunelli & Poggio ‘93• Lades, v.d. Malsburg et al. ‘93• Cootes, Lanitis, Taylor et al. ‘95• Amit & Geman ‘95, ‘99 • et al. Perona ‘95, ‘96, ’98, ’00, ‘03• Huttenlocher et al. ’00• Agarwal & Roth ’02
etc…
Parts and Structure Literature
![Page 87: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/87.jpg)
A B
DC
Deformations
![Page 88: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/88.jpg)
Background clutter
![Page 89: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/89.jpg)
Frontal faces
![Page 90: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/90.jpg)
Face images
![Page 91: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/91.jpg)
3D Object recognition – Multiple mixture components
![Page 92: lecture face recognitionFace Recognition • Digital photography • Surveillance • Album organization • Person tracking/id. • Emotions and expressions • Security/warfare](https://reader030.vdocument.in/reader030/viewer/2022041107/5f09b7f87e708231d4282fa7/html5/thumbnails/92.jpg)
3D Orientation Tuning
Frontal Profile
0 20 40 60 80 10050
55
60
65
70
75
80
85
90
95
100Orientation Tuning
angle in degrees
% C
orre
ct
% C
orre
ct