joint and implicit registration for face recognition dr. peng li and dr. simon j.d. prince...
TRANSCRIPT
Joint and implicit registration for face recognition
Dr. Peng Li and Dr. Simon J.D. Prince
Department of Computer ScienceUniversity College London
{p.li,s.prince}@cs.ucl.ac.uk
14:00-15:00 Tuesday, 23 June 2009
The face recognition pipeline
Matching
Probe Gallery
Keypoint registration Result
Detected face
Global approaches• Eigenfaces [Turk 1991]• Fisherfaces [Belhumeur 1997]
Local approaches• AAM [Cootes 2001]• ASM [Mahoor 2006]• EBGM [Wiskott 1997]
Distance-based approaches• Fisherfaces [Belhumeur1997]• Laplacianfaces [He2005]• KLDA [Yang2005]
Probabilistic approaches• Bayesian [Moghaddam 2000]• PLDA [Ioffe 2006, Prince 2007]
Feature extraction
Face recognition
Face detection
Original Image
The face recognition pipeline
•Extract Gabor jet around each keypoint
• Generative probabilistic model• Independent term for each keypoint
……
Matching
Probe Gallery
Keypoint registration
Original Image Result
Detected face
Feature extraction
Face recognition
Face detection
Hypothesis 1
H1: We can use the same probabilistic model for registration and recognition.
Probabilistic model
ResultKeypoint registration
Feature extraction
Face recognition
Face detection
Keypoint registration
Feature extraction
……
Matching
Probe GalleryDetected face
Original Image
Hypothesis 2: Joint Registration
Gallery Probe+
+
+
Generic eyeParticular eye
+
x
H2: We can use the gallery image to help find keypoints in the probe image.
Hypothesis 3: Implicit Registration
Probe
Posteriordistribution
+*
Hidden variable
H3: We do not need to make hard estimates of keypoint positions.
tp – keypoint position
Outline
• Background• Hypotheses• Probabilistic face recognition • Frontal face recognition
H1: Same model for registration and recognition H2: Joint registration H3: Implicit registration
• Cross-pose face recognition• Conclusion
Probabilistic linear discriminant analysis (Prince & Elder,ICCV 2007)
mean
m
Signal Noise
+ ++
xij = μ + + +Fhi Gwij ij
=
G(:,1)
G(:,2)
G(:,3)
w1j
w2j
w3j
Within-individual variationBetween-individual variation
F(:,1)
F(:,2)
h1
h2
h3
F(:,3)
i - # of identity
j - # of image
Image
xij
Independent per-pixel Gaussian noise,
Face recognition by model selection
xpxg
hg hp
Md
wg wp
– Match xpxg
hg
wg
Ms
wp
– No-Match
Observed Variables
Choose MAP model
Pr(xp, xg |Md)
Pr(xp, xg |Ms )
Observed Variables
Hidden Variables
• Xp - Probe image
• Xg - Gallery image
Hidden VariablesHidden
VariablesHidden
Variables
Methodology
Gallery Probe
+ +tp
1: Find keypoint in probe image alone by MAP2: Joint registration by MAP3: Implicit registration using probe image alone4: Joint and Implicit registration
Posterior over keypoint position
tp – keypoint position
xpxg
hg hp
wg wp
xpxg
hg
wg wp
Experimental Setting: XM2VTS Database
• Dataset – Training: First 195 identities– Test: Last 100 identities
• Gallery data: 1st image of 1st session • Probe data: 1st image of 4th session
• Feature Extraction: Gabor filter at all possible locations of 13 keypoints
Experiment 1: finding keypoints using recognition model in probe alone
Recognition
• First match identification rate• Higher is better
0 20 40 60 80 100 120 140 1600.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Subspace dimension
Co
rre
ct i
de
ntific
atio
n r
ate
Using keypoints labeled manually
Using keypoints found by MAP
Registration
• Average error of all keypoints• Lower is better
0 20 40 60 80 100 120 140 1600
0.02
0.04
0.06
0.08
0.1
Subspace dimension
Nor
mal
ized
Eul
idea
n D
ista
nce
Finding keypoints with MAP
Manually labeled by another subject
• Gallery image helps find keypoints in probe image• Localization errors are close to human labelling
0 20 40 60 80 100 120 140 1600.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Subspace dimension
Co
rre
ct id
en
tific
atio
n r
ate
Using probe image alone
Using both gallery and probe images
Experiment 2: joint registration
0 20 40 60 80 100 120 140 1600
0.02
0.04
0.06
0.08
0.1
Subspace dimension
No
rma
lize
d E
ulid
ea
n D
ista
nce
Using probe image alone
Using both probe and gallery images
Manually labeled by another subject
0 20 40 60 80 100 120 140 1600.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Subspace dimension
Co
rre
ct id
en
tific
atio
n r
ate
MAP
Marginalization
Experiment 3: implicit registration
• Marginalizing over keypoint position is better than using MAP keypoint position
Experiment 4: joint and implicit registration
• Joint and implicit registration performs best.• Comparable to using manually labeled keypoints.
0 20 40 60 80 100 120 140 1600.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Subspace dimension
Co
rre
ct id
en
tifica
tio
n r
ate
Using keypoints labeled manually
Using both images by marginalization
Using probe image by marginalizationUsing both images by MAP
Using probe image by MAP
Cross-pose face recognition using tied PLDA model (Prince & Elder, 2007)
Key idea: separate within-individual and between-individual variance at each pose
Data: XM2VTS database: with 90° pose difference. Gallery (frontal face) ↔ Probe (profile face)
Feature extraction: Gabor feature for 6 keypoints
FRONTAL IMAGE
PROFILE IMAGE
xijk = μk + + +Fkhi Gkwijk ijk
K = 1
K = 2
K – Pose Index
Experiment 5: Cross-pose face recognition and registration
• Similar results to frontal face recognition & registration• Comparable to using manually labeled keypoints.
0 10 20 30 40 50 60 700.55
0.6
0.65
0.7
0.75
0.8
Subspace dimension
Co
rre
ct id
en
tific
atio
n r
ate
Using keypoints labeled manually
Using both images by marginalization
Using probe image by marginalizationUsing both images by MAP
Using probe image by MAP
0 10 20 30 40 50 60 700.05
0.06
0.07
0.08
0.09
0.1
0.11
0.12
0.13
0.14
Subspace dimensionN
orm
aliz
ed
Eu
lide
an
Dis
tan
ce
Using probe image alone
Using both probe and gallery images
Manually labeled by another subject
Concluding Remarks
• Three hypotheses– Same model for both face registration & recognition.– Joint registration for face recognition– Implicit registration for face recognition
• All work well for both frontal & cross-pose face registration & recognition