an overview of research activities in biometricsmarcel/lectures/talks/smarcel... · human and...

1/30

Idiap Biometrics at Idiap Biometrics Face Recognition Others Directions

An overview of research activities in Biometricsat the Idiap Research Institute

Dr Sebastien MarcelSenior research scientistwww.idiap.ch/∼marcel

Idiap Research InstituteMartigny, Switzerland

www.idiap.ch

May 6, 2011

S. Marcel – EPFL LASEC May 2011 Biometrics Research at Idiap

2/30


Outline

1 The Idiap Research Institute

2 Biometric Person Recognition at Idiap

3 Biometric Person Recognition (Biometrics for short)

4 Face Recognition

5 Others Directions


3/30


The Idiap Research Institute

Independent not-for-profit research institute

• Founded in 1991

• Around 100 collaborators (> 25 countries)

• Budget: around 10 MCHF

• Centre du Parc in Martigny (2300 m2)

• 37 research programs (> 130 publications/year)

• Affiliated with EPFL (joint development plan) and Universityof Geneva

• Accredited (and co-funded) by the Swiss Federal Government,State and City, as “part of the “ETH Strategic Domain”

• Host institution of Swiss National Centre of Competence inResearch on “interactive multimodal informationmanagement” (IM2)


4/30


Idiap Research Activities

Perceptual and Cognitive SystemsSpeech processing, Natural language understanding and translation, Vision and scene analysis, Computational

cognitive science

Human and Social BehaviorFace-to-face communication analysis, Mobile media analysis, Social media analysis

Information Interfaces and PresentationUser interfaces, Multimedia information systems

Biometric Person RecognitionFace and speaker recognition, Mobile biometry, Emerging biometrics, Spoofing

Machine LearningStatistical and neural network based ML, Very large families of heuristics, Computational efficiency, targeting

real-time applications, Online learning


5/30


Outline


2 Biometric Person Recognition at IdiapResearch TeamActivities


4 Face Recognition

5 Others Directions


6/30


Research Team

Current team (3 PostDocs and 5 PhD students)Dr Christopher McCool, Anindya Roy, Cosmin

Atanasoaei, Dr Sebastien Marcel, Venkatesh

Bala Subburaman, Dr Roy Wallace, Laurent El

Shafey and Dr Andre Anjos.

+3 interns (Master level)

+2 engineers for support

Former PhD studentsDr Guillaume Heusch (EPFL PhD student 2009), Dr Agnes Just (EPFL PhD 2006), Dr Yann Rodriguez (EPFL

PhD 2006), Dr Fabien Cardinaux (EPFL PhD 2005).


7/30


Projects

European and International Projects• Trusted Biometrics under Spoofing Attacks (EU FP7 TABULA RASA project)

Goal: Develop and evaluate countermeasures to direct (spoofing) attacks in facerecognition (printed photos, replayed videos, ...), speech, fingerprint, iris, gait, ...Partners: IDIAP∗ (CH), UOULU (FI), UAM (SP), USOU (UK), UNICA (IT), EURECOM(FR), CASIA (CN), STARLAB (SP), MORPHO (FR), KEYLEMON (CH), BIOMETRY(CH), CSSC (IT).

• Mobile biometry (EU FP7 MOBIO project)Goal: Develop and evaluate face andspeaker recognition algorithms for mobiledevices.Partners: IDIAP∗ (CH), UMAN (UK),UNIS (UK), LIA (FR), BUT (CZ),UOULU (FI), VISIDON (FI).

Face recognition on the iPhone 4

Free App on the AppStore (FaceOnIt)

• Bayesian Biometrics for Forensic (EU Marie Curie project)

Projects in Switzerland• Swiss National Science Foundation: MULTI, GMface

• Hasler Foundation: CONTEXT

• CTI: Replay, A-vision


8/30


Activities

Research Topics

• Face recognition: face detection, facial feature localization,face identification/verification

• Speaker recognition

• Multi-Modal fusion: early and late fusion

Research Problems

• robust-to-illumination features for face recognition

• robust-to-noise binary features for speaker recognition

• statistical models for face recognition

• online (unsupervised) model adaptation

• spoofing


9/30


Outline




4 Face Recognition

5 Others Directions


10/30


Biometrics

Definition

Biometric person recognition (or Biometrics for short) refers to theprocess of automatically recognizing a person using:

• distinguishing behavioral patterns: gait, signature, keyboardtyping, lip movement, ...

• physiological traits: face, voice, iris, fingerprint, handgeometry, electroencephalogram (EEG), electrocardiogram(ECG), ear shape, vein, ...

A truly inter-disciplinary research field

Biometrics offer a wide range of challenging fundamental andconcrete problems in signal (image/audio) processing, computervision, pattern recognition and machine learning.


11/30


Biometrics

Challenges

• In most real life applications, the environment is not knowna-priori and the system should be fully automatic. Abiometric system has to deal with variabilities such as:

• Lighting variations,• Background noise (audio),• Alignment (head pose),• Deformation (facial expressions) or Corruption (occlusion),• Aging.

• Typically these variability are classified as:• extra-personal variabilities: variations in appearance between

different identities,• intra-personal variabilities: variations in appearance of the

same identity.


12/30


Biometrics

A General Biometric System

1 data capture,

2 segmentation (such as face detection, silence detection),

3 geometric normalization (such as alignment),

4 sample normalization (such as illumination normalization),

5 feature extraction,

6 enrollment and classification:• enrollment: building a template (or model) of an identity,• classification: identity recognition from models and features.


13/30


Biometrics

Classification

• Classification consists of attributing a label to the input dataand differs according to the specific task (closed or open setidentification, verification)

• All system provides a score Λ(X ; ΘI ) corresponding to anopinion on the probe feature set X to be the identity I .

• verification: the label is true (client) or false (impostor)• closed set identification: the label is the identity• open set identification: the label is the identity or unknown


14/30


Biometrics

Classification

• verification: given a threshold τ , the claim is accepted whenΛ(X ; ΘI ) ≥ τ and rejected when Λ(X ; ΘI ) < τ

• closed set identification: we can recognize identity I ∗

corresponding to the probe feature set X as follows

I ∗ = arg maxI

Λ(X ; ΘI ) (1)

• open set identification: the recognized identity I ∗

corresponding to the probe is found as follows

I ∗ =

{unknown if Λ(X ; ΘI ) < τ ∀ Iarg maxIΛ(X ; ΘI ) otherwise

(2)


15/30


Biometrics

To summarize

1 determine a feature extraction to produce X

2 choose a model of the features X (with parameters Θ)

3 compute a score Λ(X ; ΘI ) for classification


16/30


Outline




4 Face RecognitionFace Recognition using Local Binary PatternsFace Recognition using Statistical Generative Models

5 Others Directions


17/30


Face Recognition

A General Face Recognition System

1 segmentation (face detection),

2 geometric normalization (face alignment and cropping),

3 sample normalization (illumination normalization),

4 feature extraction,

5 classification.

FR

Face Recognition

FeatureExtraction Features DecisionClassifier

FaceImage

(geometrically normalized)

FaceImage

(photometrically normalized)

Photometric Normalisation


18/30


Face Recognition using Local Binary Patterns

A single technique can be used to address

illumination normalization, feature extraction, and classification.


19/30


Local Binary Patterns

Definition

• Original LBP operator: 3x3 kernel which summarizes the localspatial structure of an image.

83 75 126

99 95 141

91 91 100

0 0

0 10

1

1

1 binary: 00111001decimal: 57

comparisonwith the center

binary intensity

• LBPP,R : P equally spaced pixels on a circle of radius R.

• LBPu2P,R : only uniform patterns (at most two bitwise 0 to 1 or

1 to 0 transitions)

• Other variants: Improved LBP, Extended LBP.


20/30


Local Binary Patterns

Properties

• Very low computational cost

• Powerful texture descriptor

• Invariant to monotonic gray-scale transformation

LBP is becoming a very popular technique due to its simplicity andits interesting monotonic grayscale invariant property.


21/30


Face Recognition using Local Binary Patterns

Histograms of Local Binary Patterns

• LBP histograms are computed for blocks,

• classification is performed using a simple distance onhistograms.

Local Binary Patterns as an image pre-processing

• LBP is considered as a pre-processing only,

• then regular feature extraction and classification areperformed.


22/30


Statistical Generative Models

ΛI (X ; ΘI ) = P(X |ΘI )

Simple to complex models:

• Gaussian Mixture Models (GMM) [Sanderson:2003],

• 1D Hidden Markov Models (1D-HMM) [Eickeler:2000],

• Pseudo-2D Hidden Markov Models (P2D-HMM)[Nefian:1999],

• Bayesian Networks (BNface) [Heusch:2009].


23/30


Statistical Generative Models

using local (parts-based) feature extraction

Input Image Image Blocks

Features (DCT) from Blocks

the face image is decomposed into parts (blocks)

as opposed to holistic feature extraction

the face image is processed as a whole


24/30


Outline




4 Face Recognition

5 Others DirectionsSpeaker RecognitionSpoofing in Face Recognition


25/30


Speaker Recognition

Speaker Recognition using Binary Slice Classifiers


26/30


Speaker Recognition using Binary Slice Classifiers

An example of slices

20 40 60 80 100 120

−1

−0.5

0

0.5

1

k

X(k)

(a)

0.63

ki,2

= 60ki,1

= 9

( ki,1

) = X ( ki,2

) = X 0.03 X

Hi

Spectral vector,Slice basis vector,

−1

0

1

Li

(b)

Li = 0.60

θi = 0.30

−1

0

1

fi (L

i )(c)

fi ( L

i ) = 1

20 40 60 80 100 120

−1

−0.5

0

0.5

1

k

X(k)

0.46

(d)

( ki,1

) = ( ki,2

) = X X 0.06Hi

ki,1

= 40ki,2

= 5

Spectral vector,Slice basis vector,

X

−1

0

1

Li(e)

θi = 0.1

Li = − 0.40

−1

0

1

fi (L

i )

fi ( L

i ) = −1

(f)


27/30


Vulnerabilities to Attacks

Direct and indirect AttacksConventional biometric systems, such as fingerprint or face recognition, are vulnerable to attacks. Generally, twotypes of attacks are considered:

1 indirect attacks due to intruders, such as cyber-criminal hackers, and

2 direct (spoofing) attacks, where a person tries to masquerade as another one by falsifying data and therebygaining an illegitimate advantage.


28/30


Vulnerabilities to Attacks

A spoofing attack on a face recognition system

Possible spoofing attacks

• printed photo (normal or photo quality)

• displayed photo on a mobile phone (iPhone) or a tablet (iPad)

• replayed video on a mobile phone (iPhone) or a tablet (iPad)

• 2D or 3D mask


29/30


Counter Measures to Spoofing Attacks

Counter Measures to Spoofing Attacks

• printed photo: texture ?

• displayed photo on a mobile phone (iPhone) or a tablet(iPad): eye blinking ?

• replayed video on a mobile phone (iPhone) or a tablet (iPad):any spatio-temporal patterns ?

• 2D or 3D mask: any spatio-temporal patterns ?


30/30


The End

Thanks for listeningQuestions ?


an overview of research activities in biometricsmarcel/lectures/talks/smarcel... · human and...

Documents