classifiers for template recognition face recognitionramani/cmsc426/...facerec.pdf · classifiers...
TRANSCRIPT
![Page 1: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/1.jpg)
Classifiers for Template RecognitionReading: Chapter 22 (skip 22.3)
Face Recognition
Slide credits for this class: David Lowe, Frank Dellaert, Forsyth & Ponce, Paul Viola, Christopher Rasmussen
A. Roth for face recognition
• Examine each window of an image
• Classify object class within each window based on a training set images
![Page 2: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/2.jpg)
Classification
• Idea: we are taught to recognize objects, motions, textures … etc. by being presented examples
• How do we use this idea to construct machine based classifiers
• Previous classes we saw some approaches that did template matching
• Today extend this idea further and discuss classification of objects using features
• Very important area of computational science/statistics– Techniques are used in diverse areas such as vision,
audition, credit scores, automatic diagnosis, DNA matching ….
![Page 3: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/3.jpg)
Example: A Classification Problem
• Categorize images of fish—say, “Atlantic salmon” vs. “Pacific salmon”
• Use features such as length, width, lightness, fin shape & number, mouth position, etc.
• Steps1. Preprocessing (e.g., background
subtraction)2. Feature extraction 3. Classification
example from Duda & Hart
![Page 4: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/4.jpg)
Some errors may be inevitable: the minimumrisk (shaded area) is called the Bayes risk
Bayes Risk
Probability density functions (area under each curve sums to 1)
![Page 5: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/5.jpg)
Finding a decision boundary is not the same asmodeling a conditional density.
Discriminative vs Generative Models
![Page 6: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/6.jpg)
Loss functions in classifiers
• Loss– some errors may be more expensive than others
• e.g. a fatal disease that is easily cured by a cheap medicine with no side-effects -> false positives in diagnosis are better than false negatives
– We discuss two class classification: L(1->2) is the loss caused by calling 1 a 2
• Total risk of using classifier s
![Page 7: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/7.jpg)
Histogram based classifiers
• Use a histogram to represent the class-conditional densities– (i.e. p(x|1), p(x|2), etc)
• Advantage: Estimates converge towards correct values with enough data
• Disadvantage: Histogram becomes big with high dimension so requires too much data– but maybe we can assume feature independence?
![Page 8: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/8.jpg)
Example Histograms
![Page 9: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/9.jpg)
Kernel Density Estimation
• Parzen windows: Approximate probability density by estimating local density of points (same idea as a histogram)– Convolve points with window/kernel function (e.g., Gaussian)
using scale parameter (e.g., sigma)
from Hastie et al.
![Page 10: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/10.jpg)
Density Estimation at Different Scales
from Duda et al.
• Example: Density estimates for 5 data points with differently-scaled kernels
• Scale influences accuracy vs. generality (overfitting)
![Page 11: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/11.jpg)
Example: Kernel Density Estimation Decision Boundaries
Smaller Larger from Duda et al.
![Page 12: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/12.jpg)
Application: Skin Colour Histograms• Skin has a very small range of (intensity independent)
colours, and little texture– Compute colour measure, check if colour is in this
range, check if there is little texture (median filter)– Get class conditional densities (histograms), priors from
data (counting)
• Classifier is
![Page 13: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/13.jpg)
Skin Colour Models
Skin chrominance points Smoothed, [0,1]-normalized
courtesy of G. Loy
![Page 14: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/14.jpg)
Skin Colour Classification
courtesyof G. Loy
![Page 15: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/15.jpg)
Figure from “Statistical color models with application to skin detection,” M.J. Jones and J. Rehg, Proc. Computer Vision and Pattern Recognition, 1999 copyright 1999, IEEE
Results
![Page 16: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/16.jpg)
Figure from “Statistical color models with application to skin detection,” M.J. Jones and J. Rehg, Proc. Computer Vision and Pattern Recognition, 1999 copyright 1999, IEEE
ROC Curves(Receiver operating characteristics)
Plots trade-off between false positives and false negatives
![Page 17: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/17.jpg)
Nearest Neighbor Classifier
• Assign label of nearest training data point to each test data point
Voronoi partitioning of feature space for 2-category 2-D and 3-D data
from Duda et al.
![Page 18: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/18.jpg)
• For a new point, find the k closest points from training data• Labels of the k points “vote” to classify• Avoids fixed scale choice—uses data itself (can be very
important in practice)• Simple method that works well if the distance measure
correctly weights the various dimensions
K-Nearest Neighbors
from Duda et al.
k = 5 Example density estimate
![Page 19: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/19.jpg)
Face RecognitionFace Recognition
IntroductionIntroductionFace recognition algorithmsFace recognition algorithmsComparisonComparisonShort summary of the presentationShort summary of the presentation
![Page 20: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/20.jpg)
IntroductionIntroduction
Why we are interested in face recognition?Why we are interested in face recognition?Passport control at terminals in airportsPassport control at terminals in airportsParticipant identification in meetingsParticipant identification in meetingsSystem access controlSystem access controlScanning for criminal personsScanning for criminal persons
![Page 21: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/21.jpg)
Face Recognition AlgorithmsFace Recognition Algorithms
In this presentation are introducedIn this presentation are introducedEigenfacesEigenfacesF
![Page 22: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/22.jpg)
EigenfacesEigenfaces
Developed in 1991 by Developed in 1991 by M.TurkM.TurkBased on PCABased on PCARelatively simpleRelatively simpleFastFastRobustRobust
![Page 23: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/23.jpg)
EigenfacesEigenfaces
PCA seeks directions that are efficient for PCA seeks directions that are efficient for representing the datarepresenting the data
efficientnot efficient
Class A
Class B
Class A
Class B
![Page 24: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/24.jpg)
EigenfacesEigenfaces
PCA maximizes the total scatter
scatter
Class A
Class B
![Page 25: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/25.jpg)
EigenfacesEigenfaces
PCA reduces the dimension of the dataPCA reduces the dimension of the dataSpeeds up the computational timeSpeeds up the computational time
![Page 26: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/26.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
AssumptionsAssumptionsImages with W Images with W ×× H=NH=N22
M is the number of images in the databaseM is the number of images in the databaseP is the number of persons in the databaseP is the number of persons in the database
![Page 27: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/27.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
The databaseThe database
2
1
2
N
bb
b
=
M
2
1
2
N
cc
c
=
M
2
1
2
N
dd
d
=
M
2
1
2
N
ee
e
=
M
2
1
2
N
aa
a
=
M
2
1
2
N
ff
f
=
M
2
1
2
N
gg
g
=
M
2
1
2
N
hh
h
=
M
![Page 28: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/28.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
We compute the average faceWe compute the average face
2 2 2
1 1 1
2 2 21 , 8
N N N
a b ha b h
m where MM
a b h
+ + + + + + = = + + +
L
Lr
M M M
L
![Page 29: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/29.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Then subtract it from the training facesThen subtract it from the training faces
2 2 2 2 2 2 2 2
2 2
1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2
1 1 1 1
2 2
, , , ,
,
m m m m
N N N N N N N N
m m
N N
a m b m c m d ma m b m c m d m
a b c d
a m b m c m d m
e m f me m f
e f
e m
− − − − − − − − = = = = − − − −
− − − = = −
r rr r
M M M M M M M M
rr
M M
2 2 2 2 2 2
1 1 1 1
2 2 2 2 2 2, ,m m
N N N N N N
g m h mm g m h m
g h
f m g m h m
− − − − − = = − − −
rr
M M M M M M
![Page 30: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/30.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Now we build the matrix which is Now we build the matrix which is NN22 by by MM
The covariance matrix which is The covariance matrix which is NN22 by by NN22
m m m m m m m mA a b c d e f g h = r r r rr r r r
Cov AAΤ=
![Page 31: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/31.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Find eigenvalues of the covariance matrixFind eigenvalues of the covariance matrixThe matrix is very largeThe matrix is very largeThe computational effort is very bigThe computational effort is very big
We are interested in at most We are interested in at most MM eigenvalueseigenvaluesWe can reduce the dimension of the matrixWe can reduce the dimension of the matrix
![Page 32: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/32.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Compute another matrix which is Compute another matrix which is MM by by MM
Find the Find the MM eigenvalues and eigenvectorseigenvalues and eigenvectorsEigenvectors of Eigenvectors of CovCov andand L L are equivalentare equivalent
Build matrix Build matrix V V from the eigenvectors offrom the eigenvectors of LL
L A AΤ=
![Page 33: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/33.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Eigenvectors of Eigenvectors of CovCov are linear combination are linear combination of image space with the eigenvectors of of image space with the eigenvectors of LL
Eigenvectors represent the variation in the Eigenvectors represent the variation in the facesfaces
U AV=
![Page 34: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/34.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Compute for each face its projection onto Compute for each face its projection onto the face spacethe face space
Compute the thresholdCompute the threshold
( ) ( ) ( ) ( )( ) ( ) ( ) ( )
1 2 3 4
5 6 7 8
, , , ,
, , ,
m m m m
m m m m
U a U b U c U d
U e U f U g U h
Τ Τ Τ Τ
Τ Τ Τ Τ
Ω = Ω = Ω = Ω =
Ω = Ω = Ω = Ω =
r rr r
r rr r
1 max , 1..2 i j for i j Mθ = Ω −Ω =
![Page 35: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/35.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
To recognize a faceTo recognize a face
Subtract the average face from itSubtract the average face from it
2
1
2
N
rr
r
=
M
2 2
1 1
2 2m
N N
r mr m
r
r m
− − = −
r
M M
![Page 36: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/36.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Compute its projection onto the face spaceCompute its projection onto the face space
Compute the distance in the face space Compute the distance in the face space between the face and all known facesbetween the face and all known faces
( )mU rΤΩ =r
22 1..i i for i Mε = Ω−Ω =
![Page 37: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/37.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Reconstruct the face from eigenfacesReconstruct the face from eigenfaces
Compute the distance between the face Compute the distance between the face and its reconstructionand its reconstruction
s U= Ωr
22mr sξ = −r r
![Page 38: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/38.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Distinguish betweenDistinguish betweenIf then itIf then it’’s not a faces not a faceIf then itIf then it’’s a new faces a new faceIf then itIf then it’’s a known faces a known face
ξ θ≥, ( 1.. )iand i Mξ θ ε θ< ≥ =
min iandξ θ ε θ< <
![Page 39: Classifiers for Template Recognition Face Recognitionramani/cmsc426/...FaceRec.pdf · Classifiers for Template Recognition Reading: Chapter 22 (skip 22.3) Face Recognition Slide credits](https://reader034.vdocument.in/reader034/viewer/2022042913/5f4bd09331b63771a676d2b8/html5/thumbnails/39.jpg)
Eigenfaces, the algorithmEigenfaces, the algorithm
Problems with eigenfacesProblems with eigenfacesDifferent illuminationDifferent illuminationDifferent head poseDifferent head poseDifferent alignmentDifferent alignmentDifferent facial expressionDifferent facial expression