HUMAN RECOGNITION BASED ON FACIAL PROFILE

AND EARSFırat Onur Alsaran, Neşe Alyüz, Melike Esma İlter,

Mürsel Taşgın, Ahmet Burak Yoldemir

CmpE 58Z Term Project

EAR RECOGNITION USING LOCAL BINARY PATTERNS

Ahmet Burak Yoldemir

Motivation

Ear biometrics has several advantages over complete face

Facial biometrics may fail due to: Expressions Cosmetics Hair styles Growth of facial hair

Ears are affected very little from such changes

Ear database

448 ear images are manually cropped from profile images of CMU Multi-PIE Database

Only left ears are used There are 4 ear images of 112

people Illumination conditions of these 4

images are all different

Samples from the database

Person 1:

Person 2:

High illumination variance!

First attempts

Filter bank approaches are applied firstAngular radial

transformGaborfilters

Leung-Malikfilters

Schmidfilters

First attempts

Filter bank approaches are applied firstAngular radial

transformGaborfilters

Leung-Malikfilters

Schmidfilters

First attempts

Filter bank approaches are applied firstAngular radial

transformGaborfilters

Leung-Malikfilters

Schmidfilters

First attempts

Filter bank approaches are applied firstAngular radial

transformGaborfilters

Leung-Malikfilters

Schmidfilters

Illumination tolerance

None of the filter bank approaches is able to tolerate illumination changes, as they have fixed bases

A grayscale invariant texture measure: Local Binary Patterns

Local binary patterns - Advantages Tolerance against illumination

changes Computational simplicity A compact description of the image

Local binary patterns - Example

Local binary patterns

After obtaining LBP codes, a histogram of these codes is obtained using 256 bins

This histogram is actually a histogram of micro-patterns

The result is a 256 dimensional feature vector of an ear image

Local binary patterns

LBP method is very sensitive to high frequency components

A slight noise can change the ordering of the pixel values in a neighborhood, which results in a different micro-pattern

To prevent this, images are filtered with a Gaussian kernel of 5x5 before finding micro-patterns

Recognition step

Euclidean distance between these feature vectors is used as the (dis)similarity measure

A similarity matrix is formed using these distances

Multi-presentation approach

To increase recognition performance, multi-presentation approach is adopted

Each ear is represented using 2 images, verification is accomplished by taking 2 ear images of the user

Mean and max rules are applied to fuse the scores

Results – Without Gaussian filtering

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

FMR

GA

R

Receiver Operating Characteristic Curve (not filtered)

OriginalMulti-presentation (mean)Multi-presentation (max)

Method EER (%)

Original 32.19MP (max) 14.73MP(mean

) 1.77

Results – With Gaussian filtering

Method EER (%)

Original 13.18MP (max) 5.43MP(mean

) 1.14

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

FMR

GA

R

Receiver Operating Characteristic Curve (filtered)

OriginalMulti-presentation (mean)Multi-presentation (max)

FACE PROFILE MATCHING

Mürsel Taşgın

Facial Profile recognition

Motivation Facial profile images can be collected

from side cameras Computation complexity is lower Complementary solution for face

recognition

Profile Database

448 profile photos from Multi-PIE database

112 subjects, each having 4 photos

Facial profiles are extracted manually in the first place

Facial Profile Registration

174

36

1 2 Rotate 90º CW 3Extract profile Edge detection 4

56 Scale and move to top (nose at the center)

Chin &

nose detection using gradient of image

Nose at the center and touching top

0 50 100 150 200 250 300 350 400-40

-35

-30

-25

-20

-15

-10

-5

0

5

Histogram representation (image to function) gradient

Facial Profile Registration (cont.) Edge detection(Sobel) is used to convert black-white

profile image to a histogram function

Profile line is decreased to a single pixel white line

Nose is the highest point in the histogram

Chin point is detected using gradient of histogram and image-filling function of Matlab:

If gradient of the image changes sharply at chin area, it is marked as chin point

If image-fill function fills in the chin area then the end point is marked as chin

174

36

21014

0 50 100 150 200 250 300 350 400-16

-14

-12

-10

-8

-6

-4

-2

0

2

4

lips Image-filling detects lips, so use gradient to find chin

Facial Profile Matching (Histogram Matching)

Facial profiles are represented as histogram functions.

After registration, pointwise distance is measured:

Difference between points are summed over all points

Other metrics are available as well: Bhattacharyya distance

• White line is profile-1

• Red line is profile-2

• Green vertical lines are distances

MULTI-BIOMETRIC FUSION OF FACIAL PROFILE AND

EAR

Neşe Alyüz

Motivation

Multiple biometric sources can provide better performance

Ear and Facial Profile biometrics can be acquired simultaneously

Instead of using a single modality of ear or profile, apply fusion

Most common fusion level: score level

Heterogeneous Scores –> score normalization is important

Score Normalization Techniques

Min-max normalization Z-Score normalization Median Absolute Deviation (MAD)

normalization Tanh normalization

Min-max Normalization Best suited for the case where bounds

are known Shift scores into range [0 1] Given a set of matching scores: {sk} Normalized scores:

Original distribution is kept, only scaling

When bounds are estimated, not robust to outliers

Z-score Normalization Performs well if prior knowledge is

available Mean and standard deviation are used Given a set of matching scores: {sk} Normalized scores:

Original distribution is not retainedDoes not guarantee a common numerical rangeWhen mean and std are estimated, very

sensitive to outliers

Median Absolute Deviation (MAD) Normalization Median and MAD are insensitive to

outliers and to points in the extreme tails of the distribution

MAD normalization benefits from this fact

Normalized scores:

where MAD = median(|sk - median|)

Median and MAD have low efficiencies When score distribution is not Gaussian, poor estimatesInput distribution is not retainedNormalized scores are not in a common range

Tanh Normalization

Robust to outliers Highly efficient Normalized scores:

Tanh distribution: normalized genuine scores has a mean of 0.05 and std of ~o.o1.

Determines the spread of genuine scores

Score Fusion Techniques

MAX rule MEAN rule SUM rule PRODUCT rule

Evaluated on scores that are normalized with different approaches

Experimental Results

Initial Results on Similarity matrices of Assignment #3: Face and Fingerprint biometrics

40 subjects with 8 sample/subject SMs: 320x320 similarity matrices Enrollment: 1 sample/subject for

each bimetric

Experimental Results - EERs

Fusion MAX MEAN SUM PRODUCTNo Norm. 8.58 8.27 8.27 14.01Min-max 14.87 8.64 8.64 8.32Z-score 8.15 7.89 7.89 20.42MAD 7.88 7.86 7.86 18.58Tanh 7.84 7.61 7.61 7.57

Individual Modalities EERsFace 12.09Fingerprint 21.76

Experimental Results - TODO

Fusion MAX MEAN SUM PRODUCTNo Norm.Min-max Z-scoreMADTanh

Individual Modalities EERsFace Profile #1Face Profile #2Ear