PhD Thesis

Biometrics Science studying measurements and

statistics of biological data Most relevant application: id.

recognition

2

Why Facial Biometrics ? Most intuitive way of identification Socially and culturally accepted

worldwide It may work without collaboration

2006

43.6 %

19.2 %

2001

3

Facial Biometrics Challenges ahead

Less accurate than iris and fingerprint

Problems with uncontrolled environments (illumination, viewpoint…) Best system

AverageFully automatic

4

Active Shape Models

Automatic training from examples

User-defined template based on landmarks

Model-based parametrization

Generative models5

T.F. Cootes, C. J, Taylor, D.H. Cooper, J. Graham (1995)Computer Vision and Image Understanding, 61(1):38–59

This thesis… Focus on 3 contributions to

ASMs on relevant aspects for facial feature localization: More accurate

segmentation invariant to in-plane rotations

Add robustness to out-of-plane rotations

Estimate the Reliability of the segmentation

1

2

3

4

6

ASM: Construction of the model Face outlines based on landmarks

Shape statistics to learn spatial relations

Texture statistics for image search

Landmarked Training Set

Local texture statistics

Shape statistics PDM

IIMs

1

7

Point Distribution Model1.- The input shapes are

aligned to remove scale, translation and rotation effects.

1 1 2 2, , , , ... , 1TL L

i i i i i i ix y x y x y i N u

iv iui i i is u R v t

Image Coordinates Model Coordinates

11

8

Point Distribution Model2.- Principal Component Analysis (PCA) on the

aligned shapes

iijji

iT

i

j

N

i

T

ii

N

ii

diagSN

S

N

Φbuub

uuΦb

ΛΛΦΦ

uuuu

uu

)(

11

1

1

1

(2L)-space representationPCA-space

representation

11

9

Point Distribution Model (PDM)

iijji Φbuub

• Can determine valid shapes

• Can get closest valid shape

• Introduces a representation error

11

10

Point Distribution Model (PDM)

iijji Φbuub

More specific

More general

11

11

PDM: Modes of variationVariation from 1st Principal Component

11

12

PDM: Modes of variation

Variation from 2nd Principal Component

11

13

ASM: Local Texture Statistics (1)

First order derivatives of the pixel intensity For each landmark Sampled perpendicularly to the contour

1:

k

kj

jigNormalized

i-th landmark

0 2 4 6 8 10 12 14-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

11

14

ASM: Local Texture Statistics (2) Second order statistics for each landmark

N

sisi

T

isii

N

ssii

N

N

1,,

1,

11

1

ggggG

gg

i-th landmark

0 2 4 6 8 10 12 14-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0 2 4 6 8 10 12 14-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

11

15

ASM: Model Matching1. The average shape is placed on the image,

roughly matching the face position

11

16

2. Displacement of each landmark to minimize the Mahalanobis distance to the mean profil

3. Apply shape model restrictions

ASM: Model MatchingSteps 2 and 3 are repeated a fixed number of iterations at different resolutions, increasing detail

11

17

ASM: Model Matching11

18

ttjjt

tht

Tt iterationt

Φbuub

uuΦb

1

),(

ASM: Model Matching11

19

ASM: Complex textures Several factors modify facial

appearance beard, hair cut, glasses, teeth.

The distribution of the normalized gradient is often non Gaussian nor unimodal.

11

20

ASM: Complex textures11

21

Optimal Features ASM

Texture description based on Taylor series

Grids centered at the landmarks for local analysis

Non linear classifier (kNN) for inside-outside labeling

0

00

0)(

)(!)()(

xx

xxnxfxf

n

n

inside

outside

1 2

22

B. van Ginneken, A.F. Frangi, J.J. Staal, B.M. ter Haar Romeny, and M.A. Viergever (2002)IEEE Transactions on Medical Imaging, 21(8):924–933