![Page 1: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/1.jpg)
HUMAN RECOGNITION BASED ON FACIAL PROFILE
AND EARSFırat Onur Alsaran, Neşe Alyüz, Melike Esma İlter,
Mürsel Taşgın, Ahmet Burak Yoldemir
CmpE 58Z Term Project
![Page 2: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/2.jpg)
EAR RECOGNITION USING LOCAL BINARY PATTERNS
Ahmet Burak Yoldemir
![Page 3: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/3.jpg)
Motivation
Ear biometrics has several advantages over complete face
Facial biometrics may fail due to: Expressions Cosmetics Hair styles Growth of facial hair
Ears are affected very little from such changes
![Page 4: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/4.jpg)
Ear database
448 ear images are manually cropped from profile images of CMU Multi-PIE Database
Only left ears are used There are 4 ear images of 112
people Illumination conditions of these 4
images are all different
![Page 5: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/5.jpg)
Samples from the database
Person 1:
Person 2:
High illumination variance!
![Page 6: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/6.jpg)
First attempts
Filter bank approaches are applied firstAngular radial
transformGaborfilters
Leung-Malikfilters
Schmidfilters
![Page 7: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/7.jpg)
First attempts
Filter bank approaches are applied firstAngular radial
transformGaborfilters
Leung-Malikfilters
Schmidfilters
![Page 8: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/8.jpg)
First attempts
Filter bank approaches are applied firstAngular radial
transformGaborfilters
Leung-Malikfilters
Schmidfilters
![Page 9: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/9.jpg)
First attempts
Filter bank approaches are applied firstAngular radial
transformGaborfilters
Leung-Malikfilters
Schmidfilters
![Page 10: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/10.jpg)
Illumination tolerance
None of the filter bank approaches is able to tolerate illumination changes, as they have fixed bases
A grayscale invariant texture measure: Local Binary Patterns
![Page 11: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/11.jpg)
Local binary patterns - Advantages Tolerance against illumination
changes Computational simplicity A compact description of the image
![Page 12: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/12.jpg)
Local binary patterns - Example
![Page 13: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/13.jpg)
Local binary patterns
After obtaining LBP codes, a histogram of these codes is obtained using 256 bins
This histogram is actually a histogram of micro-patterns
The result is a 256 dimensional feature vector of an ear image
![Page 14: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/14.jpg)
Local binary patterns
LBP method is very sensitive to high frequency components
A slight noise can change the ordering of the pixel values in a neighborhood, which results in a different micro-pattern
To prevent this, images are filtered with a Gaussian kernel of 5x5 before finding micro-patterns
![Page 15: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/15.jpg)
Recognition step
Euclidean distance between these feature vectors is used as the (dis)similarity measure
A similarity matrix is formed using these distances
![Page 16: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/16.jpg)
Multi-presentation approach
To increase recognition performance, multi-presentation approach is adopted
Each ear is represented using 2 images, verification is accomplished by taking 2 ear images of the user
Mean and max rules are applied to fuse the scores
![Page 17: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/17.jpg)
Results – Without Gaussian filtering
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
FMR
GA
R
Receiver Operating Characteristic Curve (not filtered)
OriginalMulti-presentation (mean)Multi-presentation (max)
Method EER (%)
Original 32.19MP (max) 14.73MP(mean
) 1.77
![Page 18: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/18.jpg)
Results – With Gaussian filtering
Method EER (%)
Original 13.18MP (max) 5.43MP(mean
) 1.14
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
FMR
GA
R
Receiver Operating Characteristic Curve (filtered)
OriginalMulti-presentation (mean)Multi-presentation (max)
![Page 19: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/19.jpg)
FACE PROFILE MATCHING
Mürsel Taşgın
![Page 20: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/20.jpg)
Facial Profile recognition
Motivation Facial profile images can be collected
from side cameras Computation complexity is lower Complementary solution for face
recognition
![Page 21: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/21.jpg)
Profile Database
448 profile photos from Multi-PIE database
112 subjects, each having 4 photos
Facial profiles are extracted manually in the first place
![Page 22: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/22.jpg)
Facial Profile Registration
174
36
1 2 Rotate 90º CW 3Extract profile Edge detection 4
56 Scale and move to top (nose at the center)
Chin &
nose detection using gradient of image
Nose at the center and touching top
0 50 100 150 200 250 300 350 400-40
-35
-30
-25
-20
-15
-10
-5
0
5
Histogram representation (image to function) gradient
![Page 23: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/23.jpg)
Facial Profile Registration (cont.) Edge detection(Sobel) is used to convert black-white
profile image to a histogram function
Profile line is decreased to a single pixel white line
Nose is the highest point in the histogram
Chin point is detected using gradient of histogram and image-filling function of Matlab:
If gradient of the image changes sharply at chin area, it is marked as chin point
If image-fill function fills in the chin area then the end point is marked as chin
174
36
21014
0 50 100 150 200 250 300 350 400-16
-14
-12
-10
-8
-6
-4
-2
0
2
4
lips Image-filling detects lips, so use gradient to find chin
![Page 24: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/24.jpg)
Facial Profile Matching (Histogram Matching)
Facial profiles are represented as histogram functions.
After registration, pointwise distance is measured:
Difference between points are summed over all points
Other metrics are available as well: Bhattacharyya distance
• White line is profile-1
• Red line is profile-2
• Green vertical lines are distances
![Page 25: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/25.jpg)
MULTI-BIOMETRIC FUSION OF FACIAL PROFILE AND
EAR
Neşe Alyüz
![Page 26: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/26.jpg)
Motivation
Multiple biometric sources can provide better performance
Ear and Facial Profile biometrics can be acquired simultaneously
Instead of using a single modality of ear or profile, apply fusion
Most common fusion level: score level
Heterogeneous Scores –> score normalization is important
![Page 27: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/27.jpg)
Score Normalization Techniques
Min-max normalization Z-Score normalization Median Absolute Deviation (MAD)
normalization Tanh normalization
![Page 28: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/28.jpg)
Min-max Normalization Best suited for the case where bounds
are known Shift scores into range [0 1] Given a set of matching scores: {sk} Normalized scores:
Original distribution is kept, only scaling
When bounds are estimated, not robust to outliers
![Page 29: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/29.jpg)
Z-score Normalization Performs well if prior knowledge is
available Mean and standard deviation are used Given a set of matching scores: {sk} Normalized scores:
Original distribution is not retainedDoes not guarantee a common numerical rangeWhen mean and std are estimated, very
sensitive to outliers
![Page 30: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/30.jpg)
Median Absolute Deviation (MAD) Normalization Median and MAD are insensitive to
outliers and to points in the extreme tails of the distribution
MAD normalization benefits from this fact
Normalized scores:
where MAD = median(|sk - median|)
Median and MAD have low efficiencies When score distribution is not Gaussian, poor estimatesInput distribution is not retainedNormalized scores are not in a common range
![Page 31: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/31.jpg)
Tanh Normalization
Robust to outliers Highly efficient Normalized scores:
Tanh distribution: normalized genuine scores has a mean of 0.05 and std of ~o.o1.
Determines the spread of genuine scores
![Page 32: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/32.jpg)
Score Fusion Techniques
MAX rule MEAN rule SUM rule PRODUCT rule
Evaluated on scores that are normalized with different approaches
![Page 33: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/33.jpg)
Experimental Results
Initial Results on Similarity matrices of Assignment #3: Face and Fingerprint biometrics
40 subjects with 8 sample/subject SMs: 320x320 similarity matrices Enrollment: 1 sample/subject for
each bimetric
![Page 34: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/34.jpg)
Experimental Results - EERs
Fusion MAX MEAN SUM PRODUCTNo Norm. 8.58 8.27 8.27 14.01Min-max 14.87 8.64 8.64 8.32Z-score 8.15 7.89 7.89 20.42MAD 7.88 7.86 7.86 18.58Tanh 7.84 7.61 7.61 7.57
Individual Modalities EERsFace 12.09Fingerprint 21.76
![Page 35: Human Recognition Based On Facial Profile and ears](https://reader035.vdocument.in/reader035/viewer/2022062222/56815f24550346895dcdeef5/html5/thumbnails/35.jpg)
Experimental Results - TODO
Fusion MAX MEAN SUM PRODUCTNo Norm.Min-max Z-scoreMADTanh
Individual Modalities EERsFace Profile #1Face Profile #2Ear