kernel correlation feature analysis: a new advanced ...€¦ · 07/12/2016 · kcfa-v1 kcfa-v3. 34...
TRANSCRIPT
![Page 1: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/1.jpg)
1
Kernel Correlation Feature Analysis:A New Advanced Correlation Filter
Approach for RecognizingUncontrolled Face Image data in
FRGC - Phase II
Dr. Marios Savvides
Email: [email protected]/~msavvid
Research Assistant ProfessorElectrical & Computer Eng and CyLab
Carnegie Mellon University
![Page 2: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/2.jpg)
2
Acknowledgements
• In collaboration with Prof. VijayaKumar at CMU(we are participating in FRGC and FRVT!).
• Thanks to Dr. Jonathon Phillips and NIST forcreating these interesting face challengeproblems and pushing to create the nextgeneration face recognition algorithms!
• Very grateful for support of this research by theU.S. Technical Support Working Group (TSWG)!
![Page 3: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/3.jpg)
3
Challenges in Face Recognitionand how related to Quality?
• Pose
• Illumination
• Expression
• Occlusion
• Time lapse
• Individual factors: Gender
![Page 4: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/4.jpg)
4
Motivation• Face recognition is a challenging task due to large distortions in human
faces.• Face Recognition Grand Challenge (FRGC) program evaluates the state
of the art of current face recognition technology.• Lots and Lots of face data captured over a period of 2 years at
University of Notre Dame (>36,000 images).• Participation includes commercial face recognition vendors and
Universities.• We want to contribute our efforts to develop more robust face
recognition algorithms with large scale database such as the FRGC.• We have to design new approaches to recognizing faces with
correlation filters• Current Correlation Filter methods do not use generic training set, we
present a novel approach that uses generic set to build a generic CFbasis
![Page 5: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/5.jpg)
5
FRGC Experiments
The performance of PCA is about 12% at the False Accept Rate 0.1%. The Experiment 4 is most challenging due to uncontrolled conditions
Principal Component Analysis (PCA) Performance
ROC curve from P. Jonathan Phillips et al (CVPR 2005)
![Page 6: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/6.jpg)
6
FRGC Gallery/Target Images
Controlled (Indoor)Controlled (Indoor)
These images are what you have of the ‘criminal’suspect that you are looking for...
(we have 16,028 target images of 466 people)
![Page 7: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/7.jpg)
7
FRGC Query ImagesUncontrolled (Indoor)
These are the test images captured un uncontrolledconditions that we must be able to match against the
‘Target’ set
![Page 8: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/8.jpg)
8
more FRGC Query Images
Outdoor illuminationimages are very
challenging due toharsh cast shadows,these affect imagequality significantly.
Uncontrolled (Outdoor)
![Page 9: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/9.jpg)
9
Face Recognition GrandChallenge Dataset from NIST
Generic Training Set consisting of 222people with a total of 12,776 images
Gallery Set of 466 people(16,028) images total
Feature extraction Feature space generation
Reduced Dimensionality FeatureRepresentation of Gallery Set
16,028
Probe Set of 466 people(8,014) images total
Reduced Dimensionality FeatureRepresentation of Probe Set
8,014
Similarity Matching
Reduced Dimensional FeatureSpace
project project
![Page 10: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/10.jpg)
10
FRGC Experiments
The performance of PCA is about 12% at the False Accept Rate 0.1%. The Experiment 4 is most challenging due to uncontrolled
conditions. This is what we focus on….
Principal Component Analysis (PCA) Performance
ROC curve from P. Jonathan Phillips et al (CVPR 2005)
![Page 11: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/11.jpg)
11
What is PCA? What is it trying to do?• We want to find projections of data (i.e. direction vectors that we can
project the data on to) that describe the maximum variation (or capturemost of signal energy).
• Hopefully we can represent the data with a few such projection vectors.• PCA is very one of the most typically dimensionality reduction methods
used in pattern recognition.
Marios Savvides
y
x
Axis that describes thelargest variation (orscatter)
![Page 12: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/12.jpg)
12
When is PCA bad?
• What if we have 2 classes (the greenand blue dots and we want to separatethem in some feature space)?
• This is what PCA does..y
x
Axis that describes thelargest variation (orscatter)…
In this case the projection vectorcompletely smears the two classestogether, making them in-separable
![Page 13: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/13.jpg)
13
Illumination Pre-Processing• We examined different illumination algorithms and we
used one from CMU (Gross & Brajovic)
Controlled Face Images Illum-processed
![Page 14: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/14.jpg)
14
Illumination Pre-Processing
Uncontrolled Face Images (harsh illuminationconditions) Illum-processed
![Page 15: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/15.jpg)
15
Why did PCA fail in FRGC ExpIV?
• Even though PCA subspace was built using12,776 images from 222 people…..
• Final verification rate was very low (12% @0.1 FAR).
• This suggests PCA subspace could notrepresent the gallery/probe images ingenerated subspace.
• Poor discrimination ability.• Illumination pre-processing did not seem to
help PCA much.
![Page 16: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/16.jpg)
16
How about Advanced CorrelationFilter Designs?
•Advanced Correlation Filter Designs in past such asMinimum Average Correlation Energy (MACE) filterand its derivatives worked well for illuminationtolerant face recognition on CMU-Pose IlluminationExpression (PIE) database.
• How can they be used successfully in FRGC?
![Page 17: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/17.jpg)
17
Typical Enrollment Scenario for CorrelationEnrollment Scenario for CorrelationFilter RecognitionFilter Recognition
TrainingTrainingImagesImagescapturedcapturedbybycameracamera
Filter DesignFilter DesignModuleModule
CorrelationCorrelationFilter HFilter H(Template)(Template)
FrequencyFrequencyDomain arrayDomain array
FrequencyFrequencyDomain arrayDomain array
FrequencyFrequencyDomain arrayDomain array
FFTFFT
FFTFFT
FFTFFT
N x N pixelsN x N pixels N x N pixels (complex)N x N pixels (complex)
N x N pixelsN x N pixels(complex)(complex)
![Page 18: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/18.jpg)
18
RecognitionRecognition stage stage (traditional way of doing(traditional way of doingcorrelation)correlation)
Test ImageTest Imagecapturedcapturedbybycameracamera
CorrelationCorrelationFilter HFilter H(Template)(Template)
FrequencyFrequencyDomain arrayDomain array
FFTFFT
N x N pixelsN x N pixels
N x N pixelsN x N pixelsResultingResultingFrequencyFrequencyDomain arrayDomain array
IFFTIFFT
PSRPSR
![Page 19: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/19.jpg)
19
Typical Correlation Outputs from an Authentic
![Page 20: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/20.jpg)
20
Example Correlation Outputs from an Impostor
![Page 21: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/21.jpg)
21
Peak to Sidelobe Ratio (PSR)
1. Locate peak1. Locate peak
2. Mask a small2. Mask a smallpixel regionpixel region
3. Compute the mean and 3. Compute the mean and σσ in a in abigger region centered at the peakbigger region centered at the peak
PSR invariant to constant illumination changes
Match declared when PSR is large, i.e., peak must not onlybe large, but sidelobes must be small.
![Page 22: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/22.jpg)
22
What is the problem so far?
• Typical Correlation Filter Configurations donot make use of “generic” training data.
• There is no notion of “generic filter basis”• We have a Target, we build a filter to look
for it in a cluttered scene..…end ofstory…………
……or is it?
![Page 23: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/23.jpg)
23
Different Approach to Achieve Dimensionalityreduction
• Instead of finding representation coefficients,let us look at cognitive inspired approach.
• How do we discriminate people (new) peopleversus people we have seen before.
• One explanation can be we learndiscriminative features between people weknow.
• Once a new person is presented to us we findout how or who he resembles (viacorrelations) based on the discriminativefeatures of the people we know.
![Page 24: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/24.jpg)
24
From our memory of people we have seen, who does theQuery subjects look like?
Probe Test image
Generic Training Images0.1 0.8 0.05 0.3
0.1
0.8
0.05
0.3
Feature similarity Vector
•Possibly a cognitive inspired approach…
•How do you recognize or relate features of a new person you haven’t seen?
Close to 1=>very similar
Close to 0 => little similarity
![Page 25: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/25.jpg)
25
How do we do this in practice?• We train advanced correlation filters in a
discriminative way to produce an orthogonalfeature space.
• Each person has a filter which has beentrained to yield +1 correlation output for thatperson and 0 for all other people.
• Each person’s CF learns the discriminativefeatures of that person compared to everyother person we have in the database.
• Thus projection features on these CFs givesus a measure of how a new (probe) personresembles that particular person only.
![Page 26: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/26.jpg)
26
How we design Filter 1 (i.e. for person 1)
< , h1 > = 1 < , h1 > = 1 < , h1 > = 1 < , h1 > = 0 < , h1 > = 0 < , h1 > = 0 < , h1 > = 0 < , h1 > = 0 < , h1 > = 0
![Page 27: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/27.jpg)
27
How we design Filter 2
< , h2 > = 0 < , h2 > = 0 < , h2 > = 0 < , h2 > = 1 < , h2 > = 1 < , h2 > = 1 < , h2 > = 0 < , h2 > = 0 < , h2 > = 0
![Page 28: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/28.jpg)
28
How we design Filter 3
< , h3 > = 0 < , h3 > = 0 < , h3 > = 0 < , h3 > = 0 < , h3 > = 0 < , h3 > = 0 < , h3 > = 1 < , h3 > = 1 < , h3 > = 1
![Page 29: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/29.jpg)
29
How to do feature extraction
< , h1 > = c1 < , h2 > = c2 < , h3 > = c3 < , h4 > = c4 < , h5 > = c5 < , h6 > = c6 < , h7 > = c7 < , h8 > = c8 < , hN > = cN
c
Target/Query Image
![Page 30: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/30.jpg)
30
Filter Design 1…222• We design 222 filters based on the 222
generic subjects in the generic dataset.
• Each person has variable number of images.
• Correlation plane energy is minimized for allimages in the MACE filter design.
• Much more efficient dimensionalityreduction compared to PCA which needs tokeep over 2000 eigenvectors.
![Page 31: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/31.jpg)
31
Populating Similarity Matrix
Similarity between KCFA( ) and KCFA( ) cosine distance is used
Gallery im
ages
Probe Images
![Page 32: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/32.jpg)
32
Nonlinear Feature MappingRepresentations (CFs -> Kernel Cfs)
Linearly not separable problem may be separable inhigher dimensional spaces.
),()( 2xxx !
21: RR !"
x x
2x
It will be intractable to compute the high dimensional space -> Kernel tricks enable it without computing actual mapping• Kernels form a dot (inner) product >!!=< )(),(),( yxyxK
Not Linearly Separable Linearly Separable
![Page 33: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/33.jpg)
33
Experimental Results Eigenfaces (Baseline) results are provided by FRGC teams Performance measured at 0.1 % FAR (False Acceptance Rate)
0
0 . 2
0 . 4
0 . 6
0 . 8
Exp 4
P C A
G S L D A
C F A
K C F A - v 1
K C F A - v 3
![Page 34: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/34.jpg)
34
Similarity Matrix (of our poor performance algo)
(cosine distance-> thresholded 68%VR @ 0.1% FAR)
Cosine distance similarity matrix
1000 2000 3000 4000 5000 6000 7000 8000
2000
4000
6000
8000
10000
12000
14000
16000
Target
Query
![Page 35: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/35.jpg)
35
First Four classes
20 40 60 80 100 120 140 160 180 200
50
100
150
200
250
300
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
False Reject False Accept
Target Subject
Query
NoProblemo !
![Page 36: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/36.jpg)
36
A closer look….
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
TARGET
#4202
Well classified PROBEFalse Reject (some rotation,scale, due inaccurate eye-location
False AcceptWell classified PROBE
![Page 37: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/37.jpg)
37
04221d452 impostor – Out of focusblur+harsh overhead illumination
20 40 60 80 100 120
20
40
60
80
100
120
![Page 38: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/38.jpg)
38
False Accept Blocks (off-diagonal)
500 1000 1500 2000 2500 3000 3500 4000
1000
2000
3000
4000
5000
6000
7000
8000
1830 1835 1840 1845 1850 1855 1860 1865 1870 1875
7680
7700
7720
7740
7760
7780
7800
7820
7840
810 815 820 825 830 835 840 845 850 855 860
7720
7730
7740
7750
7760
7770
7780
7790
7800
7810
7820
![Page 39: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/39.jpg)
39
FA Block 1: they do look alike
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
4629 not in generic 4334 in training
Target Subject False Accepts
![Page 40: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/40.jpg)
40
20 40 60 80 100 120
20
40
60
80
100
120
810 815 820 825 830 835 840 845 850 855 860
7720
7730
7740
7750
7760
7770
7780
7790
7800
7810
7820
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
TARGET :
Eyes closed
WEAK IMPOSTORS
20 40 60 80 100 120
20
40
60
80
100
120
Closing her eyes actuallystopped False Accepts!
False Accept Block
![Page 41: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/41.jpg)
41
FA Block 2: same culprit
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
What is worrying part is….the False Accept image is male!So other domain knowledge (gender classification) can behelpful in such cases.
Target Subject False Accepts (features match up)…
![Page 42: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/42.jpg)
42
Target Subject however does a decent jobin matching with her Query images
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
5 10 15 20 25 30 35 40 45
10
20
30
40
50
60
70
80
She does pretty good
![Page 43: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/43.jpg)
43
Mismatch Case (not wellrepresented in training set?)
Target set
![Page 44: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/44.jpg)
44
Query Set
![Page 45: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/45.jpg)
45
These actually matched
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
![Page 46: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/46.jpg)
46
Problem case
4719 not ingeneric trainingset, is this why?
![Page 47: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/47.jpg)
47
Another Target with matchingproblems
![Page 48: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/48.jpg)
48
Corresponding Query images
![Page 49: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/49.jpg)
49
Some Examples of successful match(even with eye-glasses present)
![Page 50: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/50.jpg)
50
Another few
![Page 51: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/51.jpg)
51
How about a person that was notmodelled in generic and did great…?
399
10 20 30 40 50 60
10
20
30
40
50
60
70
80
![Page 52: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/52.jpg)
52
Person# 4866 Query thatalways matched
![Page 53: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/53.jpg)
53
His target set
![Page 54: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/54.jpg)
54
The ones that didn’t match
2 4 6 8 10 12 14 16 18 20 22
5
10
15
20
25
30
35
4020 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
target
query
![Page 55: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/55.jpg)
55
Another great match
Same cut lip crop issue
![Page 56: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/56.jpg)
56
Random probe
![Page 57: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/57.jpg)
57
Where it went wrong
2 4 6 8 10 12 14 16
5
10
15
20
25
30
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
20 40 60 80 100 120
20
40
60
80
100
120
Seriously sideways Illum Processing noise artifactsToo much teeth
![Page 58: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/58.jpg)
58
Another Random target class
![Page 59: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/59.jpg)
59
Some Queries that did not match
Never matched
![Page 60: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/60.jpg)
60
Where it matched
![Page 61: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/61.jpg)
61
Distance Measure using SupportVector Machines
.
.all vs. 2 classSVM
all vs. 1 classSVM
all vs. 466 classSVM
Gallery – 466 subjects (16,028controlled images) Probe – 466 subjects (8,014 uncontrolled
images)
Similarity Matrix ( 16.028 X 8.014)
Distance between
6 (gallery) vs. 3(probe)
![Page 62: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/62.jpg)
62
New Results Using distance measure using the SVM improves the
performance (KCFAv1 shown below)
FAR at 0.1%
![Page 63: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/63.jpg)
63
KCFA-SVM really improvedperformance
325
325
BEFORE: Part of similarity matrixusing cosine distance only
AFTER: Part of similarity matrixusing KCFA+SVM distance
![Page 64: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/64.jpg)
64
Other comparisons
• What if we didn’t use Correlation Filtersin CFA framework?
• Are Correlation Filters the OptimalClassifiers to use?
• Use SVM in CFA discriminativeframework and see what happens.
• Use Kernel Discriminant Analysis (KDA)in CFA discriminative framework
![Page 65: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/65.jpg)
65
Different CFA ClassifiersBenchmarks
![Page 66: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/66.jpg)
66
Final Results Using distance measure in the SVM space improve the
performance regardless of the algorithms
0
0 .5
1
NormalizedCosine
SVM space
Verf
icat
ion
Rate
s
PCA
KPCA
GSLDA
KDA
KCFA v1
KCFA v3All results are based on using the exact same
processed image data.
![Page 67: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/67.jpg)
67
Which Facial Regions are mostdiscriminative?
• Interesting question to pose…which facial partscontain most discrimination or consistency.
• Do such an analysis on FRGC is great chance tomake some observations based on largeamounts of face data
• We split FRGC data into 3 facial parts:– Eye Region– Nose Region– Mouth Region
• Do KCFA analysis on each region and analyzeperformance results.
![Page 68: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/68.jpg)
68
Facial Regions used
• Eye Region
• Nose Region
• Mouth Region
![Page 69: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/69.jpg)
69
Experiments• SVM-KCFAv1 experiments for each face region.
![Page 70: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/70.jpg)
70
Discriminative Regions
• We observe that eye-region is most discriminative(83.1%, 85% @ 0.1% FAR for KCFAv1, and KCFAv3 )
• Almost gives performance as using whole face region.
• Make intuitive sense as facial expressions changeregions around the mouth and nose much more thaneye-region.
• Fusion of KCFAv1 features from eye-region and face-region increase performance to nearly ~90% @ 0.1FAR. Now computing KCFAv3 fusion to seeperformance boost.
![Page 71: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/71.jpg)
71
What about registration errors?(not using FFTs for shift-invariance)
20 40 60 80 100 120
20
40
60
80
100
120
10 pixels shift
3 pixels shift
Effect on Performance on FRGC2? 68%->64% @0.1 FAR (cos distance)
91.3%-> ~89% @ 0.1FAR (SVM)
![Page 72: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/72.jpg)
72
Work to be done.• Must improve illumination pre-processing. Use quality measure
to weight scores.
• Pose estimation and correction, can be factored in imagequality and recognition confidence.
• Quality can also be linked to :• Image alignment: as this affects matching performance
– Robust eye-localization for scale and rotation normalization of face– Other fiducial points used for face registration.– Blur/out-of-focus
• Extract & use semantic information such as a gender classifier(and other information) to possibly reduce false matches.Mixture of experts (KCFA+?+?+?)
• Too many interesting & challenging things to do!...isn’t thisresearch field GREAT?!
![Page 73: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/73.jpg)
73
Conclusion• We can achieve ~92% using latest KCFA feature extraction methods
(on whole face).
• Eye-region most discriminative, yields 85.1% alone! (working ongetting new fusion results!)
• Substantial improvement over PCA which yields 12% @ 0.1% FAR
• Advanced Correlation Filters show superior performance yielding thebest results as a CFA feature extraction classifier compared to othermethods such as LDA, KDA, SVM.
• KCFA only extracted 222 features on all experiments, thus testingand discriminative learning is very fast and efficient!
• Our goal was to build the best “Core” matcher, now we can buildmore modules around this core matcher to improve performance:
![Page 74: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/74.jpg)
74
We are very grateful to theUnited States Technical Support
Working Group (TSWG) for fundingand supporting this research effort
Special Thanks to: Dr. David Herrington and Dr. Jim Wayman
![Page 75: Kernel Correlation Feature Analysis: A New Advanced ...€¦ · 07/12/2016 · KCFA-v1 KCFA-v3. 34 Similarity Matrix (of our poor performance algo) (cosine distance-> thresholded](https://reader036.vdocument.in/reader036/viewer/2022063018/5fdc07850ffc777dbc49d2bc/html5/thumbnails/75.jpg)
75
Questions?