![Page 1: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/1.jpg)
1
Vijayakumar Bhagavatula
Vijayakumar BhagavatulaDept. of Electrical and Computer Engineering
Carnegie Mellon UniversityPittsburgh, PA 15213
e-mail: [email protected].: (412) 268-3026
Spatial Frequency Domain Methods for Face and Iris Recognition
![Page 2: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/2.jpg)
2
Vijayakumar Bhagavatula
Acknowledgments
Funding SupportTechnology Support Working Group (TSWG), US Government.CyLab, Carnegie Mellon University
Students & ColleagueDr. Marios SavvidesChunyan XieJason Thornton
![Page 3: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/3.jpg)
3
Vijayakumar Bhagavatula
Outline
Spatial frequency domain approach (i.e., correlation filters) for biometric recognitionBiometric verification examples
FaceIris
Face Recognition Grand Challenge (FRGC) & Iris Challenge Evaluation (ICE) resultsSummary
![Page 4: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/4.jpg)
4
Vijayakumar Bhagavatula
Motivation for Spatial Frequency Domain Many biometric modalities produce images
Face Fingerprint Iris Palmprint
Most biometric verification methods work in image domain, with their success critically depending on the features chosen whereas frequency-domain methods let the data “speak for itself”.Spatial frequency domain pattern recognition proved successful in automatic target recognition (ATR) applications where the targets exhibit significant variabilityCan we use spatial frequency domain methods for biometric verification?
![Page 5: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/5.jpg)
5
Vijayakumar Bhagavatula
Pattern VariabilityFacial appearance may change due to illuminationFingerprint image may change due to plastic deformationIris pattern may change due to rotations, pupil dilation, etc.
![Page 6: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/6.jpg)
6
Vijayakumar Bhagavatula
Pattern Recognition Approaches
Statistical methods (e.g., Bayes decision theory)Model-based approachesArtificial neural networksFrequency domain methods (Correlation filters)
No need for segmentation of test imagesGraceful degradationClosed-form expressions
Segment &
Center
ExtractFeatures ClassifyInput Class
![Page 7: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/7.jpg)
7
Vijayakumar Bhagavatula
Correlation Filters
MatchNo Match
DecisionTest Image IFFT Analyze
Correlation output
FFT
Correlation Filter
Filter Design . . .Training Images
TrainingRecognition
Ref.: B.V.K. Vijaya Kumar, A. Mahalanobis & Richard D. Juday, Correlation Pattern Recognition, Cambridge University Press, November 2005.
![Page 8: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/8.jpg)
8
Vijayakumar Bhagavatula
Peak to Sidelobe Ratio (PSR)
σmeanPeakPSR −
=
1. Locate peak1. Locate peak
2. Mask a small 2. Mask a small pixel regionpixel region
3. Compute the mean and 3. Compute the mean and in a in a bigger region centered at the peakbigger region centered at the peak
PSR invariant to constant illumination changes
Match declared when PSR is large, i.e., peak must not only be large, but sidelobes must be small.
![Page 9: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/9.jpg)
9
Vijayakumar Bhagavatula
CMU Pose, Illumination and Expression (PIE) Database One face under 21 illuminations65 subjects
![Page 10: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/10.jpg)
10
Vijayakumar Bhagavatula
Train on 3, 7, 16, Train on 3, 7, 16, --> Test on 10.> Test on 10.
Match Quality = 40.95
![Page 11: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/11.jpg)
11
Vijayakumar Bhagavatula
Using the same filter as before, Match Quality = 30.60
Occlusion of Eyes
![Page 12: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/12.jpg)
12
Vijayakumar Bhagavatula
Match Quality = 22.38
Uncentered Images
![Page 13: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/13.jpg)
13
Vijayakumar Bhagavatula
ImpostorUsing someone else’s filterPSR = 4.77
![Page 14: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/14.jpg)
14
Vijayakumar Bhagavatula
Features of Correlation Filters
Shift-invariant; no need for centering the test imageGraceful degradationCan handle multiple appearances of the reference image in the test imageClosed-form solutions based on well-defined metrics
Ref: B.V.K. Vijaya Kumar, “Tutorial survey of composite filter designs for optical correlators,” Appl. Opt., Vol. 31, pp. 4773-4801, 1992.
![Page 15: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/15.jpg)
15
Vijayakumar Bhagavatula
Training ImagesThree face images used to synthesize a correlation filter and an individual eigenspace to perform verificationThe three selected training images consisted of 3 extreme cases (dark left half face, normal face illumination, dark righthalf face).
n = 3 n = 7 n = 16
![Page 16: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/16.jpg)
16
Vijayakumar Bhagavatula
0 10 20 30 40 50 60
10
20
30
40
50
60
Equal Error Rate using Individual Eigenface Subspace Method on PIE Database with No Background Illumination
Person
Equ
al E
rror R
ate
Average Equal Error Average Equal Error Rate = 30.8 %Rate = 30.8 %
Equal Error Rate for Individual Eigenspace Method
![Page 17: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/17.jpg)
17
Vijayakumar Bhagavatula
Reject Reject
AuthenticateAuthenticateThresholdThreshold
EER using Correlation Filter
![Page 18: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/18.jpg)
18
Vijayakumar Bhagavatula
Face Identification
Face recognition: given an input face image, to whom does it belong in a database?
If database contains N people where each person has 1 filter, then perform N correlations of the test image, one with each of the filters in the database.
The input is assigned to the filter class yielding the largest PSR.
![Page 19: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/19.jpg)
19
Vijayakumar Bhagavatula
Face Identification -Train with Illumination Variations
Training Images (selected for each person)
No. misclassifications % Accuracy% Accuracy
3, 7,16 0 100%1,10,16 0 100%2, 7, 16 0 100%4, 7, 13 0 100%1, 2, 7, 16 0 100%3,10,16 0 100%3, 16 0 100%
![Page 20: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/20.jpg)
20
Vijayakumar Bhagavatula
Face Identification –Train on Frontal Illumination
Training Images (selected for each person)
No. misclassifications % Accuracy% Accuracy
5,6,7,8,9,10,11,18,19,20 0 100%5, 6, 7, 8, 9, 10, 11, 12 0 100%5, 6, 7, 8, 9, 10 0 100%5, 7, 9,10 0 100%7,10,19 0 100%6,7,8 0 100%8,9,10 0 100%18,19,20 0 100%
![Page 21: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/21.jpg)
21
Vijayakumar Bhagavatula
Partial Face Identification
MACE MACE FilterFilter
MACE MACE FilterFilter
Train on 3, 7, 16 for Train on 3, 7, 16 for each personeach person
Train on 3, 7, 16 for Train on 3, 7, 16 for each personeach person
Accuracy = 100 % Accuracy = 100 %
Accuracy = 99.5 % (7 misses)Accuracy = 99.5 % (7 misses)
5 Pixels5 Pixels
30 Pixels30 Pixels
5 Pixels5 Pixels
30 Pixels30 Pixels
![Page 22: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/22.jpg)
22
Vijayakumar Bhagavatula
Partial Face Identification
Recognition Accuracy = 99.7 %
Num of mis-classifications = 4
Recognition Accuracy = 95.8 %
Num of mis-classifications = 57
15 Pixels15 Pixels
50 Pixels50 Pixels
50 Pixels50 Pixels
15 Pixels15 Pixels
Ref: M. Savvides, B.V.K. Vijaya Kumar and P.K. Khosla, “Partial Face Identification using Advanced Correlation Filter Methods,” Biometric Technology for Human Identification, SPIE Proc., Vol. 5404, April 2004.
![Page 23: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/23.jpg)
23
Vijayakumar Bhagavatula
Recognition using selected face regions
Using Training set #1 (3 Using Training set #1 (3 extreme lighting images)extreme lighting images)
Using Training set #2 (3 Using Training set #2 (3 frontal lighting images)frontal lighting images)
![Page 24: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/24.jpg)
24
Vijayakumar Bhagavatula
Iris BiometricPattern source: muscle ligaments (sphincter, dilator), and connective tissue
Inner boundary (pupil)Outer boundary
(sclera)
Sphincter ring
Dilator muscles
Advantages
Extremely unique pattern.
Remains stable over an individual’s lifetime.
![Page 25: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/25.jpg)
25
Vijayakumar Bhagavatula
First Step: Iris Segmentation
Detect iris boundaries
“Unwrap” into polar coordinates
Normalize radius
Standard segmentation procedure: 1
1 J.G. Daugman, “High Confidence Visual Recognition of Persons by a Test of Statistical Independence,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 11, pp. 1148-61, Nov. 1993.
Iris image
0 2π0
1
ρ
θ
radius
angle
Example iris mapping
Iris is mapped into a rectangle in normalized polar coordinate system.
This segmentation normalizes for scale change and pupil dilation.
All iris patterns map to the same size, which makes recognition easier.
![Page 26: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/26.jpg)
26
Vijayakumar Bhagavatula
Gabor Wavelet Iris Encoding (GWIE)
2D and 3D views of Gabor wavelet (real part)
Project onto a family of Gabor wavelet filters
Quantize phase response of each filter to 2 bits
Append bits together
Segmented iris pattern
The resulting bit vector represents the encoded iris features used for matching.
220
2200 /)(/)()(),( βθθαθθωθ −−−−−−= eeerG rri
![Page 27: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/27.jpg)
27
Vijayakumar Bhagavatula
Iris Recognition: Correlation FiltersWe use an alternative method for iris recognition, based on correlation filters. We design a filter for each iris class using a set of training images.
match
no match
FFT-1x
Correlation filter
FFT
Segmented iris pattern
Determining an iris match with a correlation filter
![Page 28: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/28.jpg)
28
Vijayakumar Bhagavatula
Experiments on CASIA Database
108 iris classes
7 images per class, collected in 2 sessions
resolution 280 by 320
collected with IR illumination
We applied correlation filters to sections of each segmented iris (corresponding to left and right sides of the iris).
We set the recognition threshold in order to compute Equal Error Rates.
![Page 29: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/29.jpg)
29
Vijayakumar Bhagavatula
Iris Verification with GWIE
Training on first image only:
Overall Equal Error Rate (EER): 4.09 %
Normalized histograms of Hamming similarities (red = imposters, blue = authentics)
Using Masek’s implementation of Daugman’s iris code algorithm
Impostors
Authentics
Libor Masek, Peter Kovesi. MATLAB Source Code for a Biometric Identification System Based on Iris Patterns. The School of Computer Science and Software Engineering, The University of Western Australia. 2003.
![Page 30: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/30.jpg)
30
Vijayakumar Bhagavatula
Iris verification with Correlation FiltersTraining on first image only: (using a single iris image for training)
Overall Equal Error Rate (EER): 0.94 %
Classes w/ complete score separation: 98 (of 108)
Normalized match score histograms (red = imposters, blue = authentics)
Impostors
Authentics
![Page 31: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/31.jpg)
31
Vijayakumar Bhagavatula
FRGC Dataset: Experiment 4
Generic Training Set consisting of 222 people with a total of 12,776 images
Gallery Set of 466 people (16,028) images total
Feature extraction Feature space generation
Reduced Dimensionality Feature Representation of Gallery Set
16,028
Probe Set of 466 people (8,014) images total
Reduced Dimensionality Feature Representation of Probe Set
8,014
Similarity Matching
Reduced Dimensional Feature Space
project project
![Page 32: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/32.jpg)
32
Vijayakumar Bhagavatula
FRGC Baseline Results
The verification rate of PCA is about 12% at False Accept Rate 0.1%.
ROC curve from P. Jonathan Phillips et al (CVPR 2005)
![Page 33: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/33.jpg)
33
Vijayakumar Bhagavatula
FRGC Expt. 4 PerformanceEigenfaces (Baseline) results provided by FRGC teamPerformance measured at 0.1 % FAR (False Acceptance Rate)
0
0 . 2
0 . 4
0 . 6
0 . 8
Ex p 4
P CAGS LDA
CFAKCFA- v 1KCFA- v 3
KCFA- v 5
72% @ 0.1FAR
![Page 34: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/34.jpg)
34
Vijayakumar Bhagavatula
Source: Jonathon P. Phillips, NIST
![Page 35: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/35.jpg)
35
Vijayakumar Bhagavatula
Source: Jonathon P. Phillips, NIST
![Page 36: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/36.jpg)
36
Vijayakumar Bhagavatula
SummaryFrequency-domain verification algorithms offer advantages
Shift-invarianceGraceful degradationClosed-form solutions
Correlation filters offer excellent performance for image biometrics (e.g., face, iris, fingerprint and palmprint).Same correlation engine can be used for multiple biometric modalities.
![Page 37: Spatial Frequency Domain Methods for Face and Iris …kumar/Bhagavatula_May2006_talk.pdfIris Recognition: Correlation Filters We use an alternative method for iris recognition, based](https://reader031.vdocument.in/reader031/viewer/2022030418/5aa537cb7f8b9a517d8cf016/html5/thumbnails/37.jpg)
37
Vijayakumar Bhagavatula
Our Current Research Directions in BiometricsFace Recognition Vendor Test (FRVT) 2006Iris Challenge Evaluation (ICE) 2006Improved face recognitionLarge-population face recognitionFace recognition from low-quality imagesImproved iris recognition Reduced-complexity (e.g., PDA, cell phone, etc.) biometric recognition algorithmsMulti-biometric fusion
We welcome enquiries from funding agencies and contractors interested in supporting us further our technology and/or commercializing it