contactless and less-constrained palmprint recognition - ph.d. presentation

Contactless and less-constrainedpalmprint recognition

Angelo Genovese

Università degli Studi di MilanoDepartment of Computer Science

via Bramante 65, I-26013 Crema (CR), [email protected]

SUPERVISOR:Prof. Vincenzo Piuri

CO-SUPERVISOR:Dr. Fabio Scotti March 18, 2014

mailto:[email protected]

Angelo Genovese – Contactless and less-constrained palmprint recognition 2

Outline

• Introduction

- Biometrics

- Unconstrained and less-constrained biometrics

Contactless and less-constrained palmprint recognition

• Overview of palmprint recognition

• Researched methods

- Contactless and less-constrained palmprint recognition

Fixed distance

Uncontrolled distance

• Experiments and results

• Conclusions and future worksMarch 18, 2014

Angelo Genovese – Contactless and less-constrained palmprint recognition

Biometrics

3

• Traditional recognition methods

- Password, smartcard

• Biometrics

- Physiological

Fingerprint

Iris

Hand geometry

Palmprint

Palmvein

Ear

ECG

DNA

- Behavioral

Voice

Gait

Signature

Keystroke

0 0.5 1 1.5 2 2.5 3 3.5 4-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Time (seconds)

x

Segnale

Immagine originale + minuzie NIST (solo per controllare se calcola

March 18, 2014


Unconstrained andless-constrained biometrics

• Unconstrained biometrics

- Uncooperative subjects

- Uncontrolled scenarios

• Less-constrained biometrics

aim at using samples captured

- Contactless

- Higher distances

- Natural light conditions

- On the move

- …March 18, 2014


Contactless and less-constrained palmprint recognition (1/3)

March 18, 2014

• In this thesis, less-constrained techniques for contactless palmprint recognitionhave been researched

• Innovative methods have been studied

- Acquisition of samples

- Processing

- Biometric matching

• Original results have been obtained

- Accurate recognition

- Robust to variations in operational conditions

- Usable



March 18, 2014

• Novelty and originality

- Much less-constrained methodthan the approaches in the literature

Fully contactless

Uncontrolled hand position, no support required

No hygiene problems

Greater user acceptability

Fast acquisition (single shot)

Robust to pose and illumination

- Low cost with respect tosimilar approaches in the literature

Only two cameras and simple illumination systems



March 18, 2014

• Less-constrained fingerprint recognitionhas been studied in previous works

- Comparison methods are standardand publicly available

• Results enabled the study ofpalmprint recognition methods

- Palmprint features are similar tofingerprint features

- Similar techniques foracquisition and processing


Possible applications ofcontactless palmprint recognition

March 18, 2014

Logical accessto terminals

Physical accessto restricted areas


Palmprint recognition:Comparison with fingerprints

• Pros:

- Low resolutions (< 200 dpi)

500 dpi needed for fingerprints

- Can be acquired in more situations

Manual workers, elder people

- User acceptability

- Multibiometric system

Combination with fingerprint, finger shape, hand shape, etc.

• Cons:

- High accuracy features not always usable (e.g., minutiae)

March 18, 2014


Palmprint recognition:Taxonomy

March 18, 2014


Palmprint recognition:Contactless vs contact

• Pros:

- Less distortion

- No dirt

- Increased user acceptability

• Cons:

- Low contrast

- Complex background

- Sensible to lighting

- Sensible to position

March 18, 2014


Palmprint recognition:3D vs 2D

• Pros:

- Robust to lighting, occlusions, noise

- Robust to spoofing attacks

- Invariant to position and distance

- Can use also 2D information

• Cons:

• Complex equipment

• Can be expensive

March 18, 2014


Palmprint recognition:State of the art (1/3)

• Contact-based 2D systems

- CCD-based scanner

- Optical device

- Flatbed scanner

• Contact-based 3D systems

- Structured light illumination

March 18, 2014



• Contactless 2D systems:

- Cameras

- Smartphones

- Webcams

• Contactless 3D systems:

- Laser scanners

March 18, 2014



March 18, 2014

• Recognition algorithms:

- Ridge based

- Line based

- Subspace based

- Statistical

- Coding based


Previous research incontactless fingerprint recognition

• The different steps in contactless fingerprint recognition have been studied

- Multiple-view acquisition

- Preprocessing

- Three-dimensional reconstruction

- Texture enhancement

- Three-dimensional matching

• Similar steps have been studiedfor palmprint recognition

March 18, 2014


Researched methods:Palmprint acquisition systems

March 18, 2014

Contactless palmprint recognition at a fixed

distance

Fully contactless, less-constrained palmprint recognition with uncontrolled distance


Contactless palmprint recognitionat a fixed distance

March 18, 2014


Acquisition

March 18, 2014

Image A Image B


3D palm reconstruction (1/2)

March 18, 2014


3D palm reconstruction (2/2)

March 18, 2014

• Point matching:

- Preliminary match in the second image using the homography mapping 𝑋’𝐵 = 𝐻 𝑋𝐴

- Point with the maximum cross-correlation value is used

𝑙 × 𝑙 squared window

Δ𝑥 × Δ𝑦 search range


Texture enhancement

March 18, 2014

• Removal of the skin tone

- Adaptive histogram equalization

- Background subtraction

• Enhancement of the details of the palm

- Logarithm

• Removal of ridges

- Smoothing


2D feature extraction and matching

March 18, 2014

• SIFT features are considered

- Robust to scale, rotation, and translations

• Alignment and refinement added to increase the robustness

- Image align

- Extraction and matching of SIFT descriptors

Euclidean distance

- Refinement based on collinearity



March 18, 2014

• Delaunay triangulation to refine the matches

- Similar groups of three points are more robust

- Computation of three-dimensional coordinates

- Delaunay triangulation

- Extraction of similar triangles

Match score

𝑋𝐴𝑓 = 𝑆𝐴𝑥 𝑥𝐴𝑓, 𝑦𝐴𝑓 ; 𝑋𝐵𝑓 = 𝑆𝐵𝑥 𝑥𝐵𝑓, 𝑦𝐵𝑓 ;

𝑌𝐴𝑓 = 𝑆𝐴𝑦 𝑥𝐴𝑓,𝑦𝐴𝑓 ; 𝑌𝐵𝑓 = 𝑆𝐵𝑦 𝑥𝐵𝑓, 𝑦𝐵𝑓 ;

𝑍𝐴𝑓 = 𝑆𝐴𝑧 𝑥𝐴𝑓, 𝑦𝐴𝑓 ; 𝑍𝐵𝑓 = 𝑆𝐵𝑧 𝑥𝐵𝑓,𝑦𝐵𝑓 ;


Fully contactless palmprint recognitionwith uncontrolled distance

March 18, 2014


Acquisition (1/2)

March 18, 2014

Image A Image B

• Special acquisition system

- Optimization of optics, illumination, distances

• Less-constrained acquisition

- Fully contactless

- Uncontrolled position

- Relaxed hand

- Palmprint must be visible

Horizontal orientation

Small rotations are tolerated


Acquisition (2/2)

March 18, 2014

• Uniform illumination

• Different setups and wavelengths studied

- Three downlights with white leds

- Four blue led bars


3D palm reconstruction andmodel normalization

March 18, 2014

• 3D reconstruction

- Point matching and triangulation

Homography

Cross-correlation

- Point cloud filtering

- Surface estimation

• 3D normalization tocompensate rotations

- Plane fitting

Palm is almost flat

- Trigonometry formulas


3D image registration andtexture enhancement

March 18, 2014

• The model is reprojected on the image plane

- Using calibration information

- Normalized position

Invariant to the acquisition positionand distance

• Texture enhancement

- Removal of the skin tone

- Enhancement of the details of the palm

- Removal of ridges



March 18, 2014

• SIFT-based alignment for horizontal rotations

- 3D features are not robust to horizontal rotations

- Extraction and matching of points

- Estimation of rotation and translation

RANSAC algorithm

• SIFT-based 2D feature extraction and matching

- Robust to uncontrolled acquisitions

- Extraction and matching of SIFT descriptors

- Refinement based on collinearity



March 18, 2014

• Delaunay triangulation to refine the matches

- Similar groups of three points are more robust

- Computation of 3D coordinates

- Delaunay triangulation

- Extraction of similar triangles

Match score


Experimental results:Accuracy of different illumination methods

March 18, 2014

• Palmprints capturedwith uncontrolleddistance

- 64 palms, 640 samples

- White light

Equal Error Rate= 4.13%

- Blue light

Equal Error Rate= 2.53%

Receiver Operating Characteristic

FMR = False Match RateFNMR = False Non-match Rate


Experimental results:Multiple comparisons

March 18, 2014

• Considered the bestof 3 comparisons

- Maximum match score

• Combination of blueand white

- Mean match score

EER = 0.08%

Fusionscheme

EER (%)FNMR @FMR= 0.05%

FNMR @FMR= 0.10%

FNMR @FMR= 0.25%

FMR@FNMR= 0.10%

FMR@FNMR= 0.25%

Mean 0.08 0.10 0.09 0.07 0.06 0.00

Receiver Operating Characteristic


Experimental results:Robustness to hand orientation

March 18, 2014

• Hand positioned with differentroll orientations

- Good tolerance


Experimental results:Robustness to illumination

March 18, 2014

• Different environmental illuminations

- Laboratory acquisition, morning light, afternoon light, artificial light

• Match scores are not affected

Illumination situation

Match scores

Genuine comparisons

Impostor comparisons

Mean Std Mean Std

Laboratory acquisition 3179.8 942.1 3.3 1.8

Morning light 2677.4 941.6 2 0.8

Afternoon light 2748.9 903.2 2 0.8

Artificial light 2770.9 876.8 2 0.8


Experimental results:Evaluation of usability and social acceptance

March 18, 2014

• Usability

- Evaluation of the quality of the samples

- Evaluation of the time needed for the acquisition

- Evaluation of users’ opinion

E.g., Is the acquisition comfortable?

• Social acceptance

- Evaluation of users’ opinion

E.g., Are you worried about hygiene issues?

E.g., Do you feel that the system attacks your privacy?


Experimental results:Comparison with the literature (1/2)

March 18, 2014

• Based on the acquisition

- Fully contactless, less-constrained acquisition

- No pegs

- No dirt, sweat, or latent impressions

- Faster acquisition, simpler setup

- Less expensive than the methods based on 3D models

• Based on the accuracy

- Better accuracy than the methods based on 3D models and uncontrolled acquisitions


Experimental results:Comparison with the literature (2/2)

March 18, 2014

ReferenceType of

acquisitionDevice

Size of dataset (palms)

EER(%)

Li et al., 2012Contact

2D

CCD-based with pegs 386 0.02

Cappelli et al., 2012 Optical device 160 < 0.01

Wang et al., 2012 Flatbed scanner 384 0.20

Li et al., 2012Contact

3DCCD-based and

projector, with pegs100 0.03

Jia et al. 2012 Contactless 2D

Mobile device 200 0.14

Tiwari et al., 2013 Ad-hoc device 602 0.06

Kanhangad et al. 2011 Contactless

3D

Laser scanner 354 0.22

ProposedMethod

Two-viewCCD-based

64 0.08


Conclusions

March 18, 2014

• Innovative palmprint recognition system

- Fully contactless, less-constrained

- Uncontrolled hand position

Based on 3D models

- Robust to orientationand illumination changes

- Fast acquisition

- Lower cost setup

- Accurate

- Good usability

- Good social acceptance


Future works

March 18, 2014

• Future developments are possible

- Real-time hand detector

Quality estimation of acquisition

- More accurate 3D models

More robust 3D matching methods

Robust 3D alignment methods

- Simultaneous acquisition with multiple illuminations

- Lower shutter time for faster acquisitions

E.g. waving hand

- Enclosed structure

Increased ease of use

Increased illumination efficiency


Publications (1/4)

March 18, 2014

Refereed international journal articles1. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Touchless Fingerprint Biometrics: a

Survey on 2D and 3D Technologies", in Journal of Internet Technology, 2014 (to appear).

Chapters in books2. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Iris segmentation: state of the art and

innovative methods", in Cross Disciplinary Biometric Systems, C. Liu, and V.K. Mago (eds.),Springer, pp. 151-182, 2012. ISBN: 978-3-642-28457-1.

Refereed papers in proceedings ofinternational conferences and workshops3. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Accurate 3D Fingerprint Virtual

Environment for Biometric Technology Evaluations and Experiment Design", in 2013 IEEE Int.Conf. on Computational Intelligence and Virtual Environments for Measurement Systems andApplications (CIVEMSA 2013), July 15 - 17, 2013.


Publications (2/4)

March 18, 2014

4. R. Donida Labati, A. Genovese, V. Piuri and F. Scotti, “Contactless Fingerprint Recognition: aNeural Approach for Perspective and Rotation Effects Reduction”, in 2013 IEEE Symposium onComputational Intelligence in Biometrics and Identity Management (CIBIM 2013), April 2013.

5. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Two-view Contactless FingerprintAcquisition Systems: a Case Study for Clay Artworks", in 2012 IEEE Workshop on BiometricMeasurements and Systems for Security and Medical Applications (BioMS 2012), pp. 1-8,September, 2012.

6. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Virtual Environment for 3-D SyntheticFingerprints", in 2012 IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems (VECIMS 2012), pp. 48-53, July, 2012.

7. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Weight Estimation from FrameSequences Using Computational Intelligence Techniques", in 2012 IEEE InternationalConference on Computational Intelligence for Measurement Systems and Applications (CIMSA2012), pp. 29-34, July, 2012.

8. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Quality Measurement of UnwrappedThree-dimensional Fingerprints: a Neural Networks Approach", in 2012 International JointConference on Neural Networks (IJCNN 2012), pp. 1123-1130, June, 2012.


Publications (3/4)

March 18, 2014

9. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Fast 3-D Fingertip Reconstruction Usinga Single Two-View Structured Light Acquisition", IEEE Workshop on Biometric Measurementsand Systems for Security and Medical Applications (BioMS 2011), pp. 1-8, September, 2011.

10. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Measurement of the principal singular pointin fingerprint images: a neural approach", in 2010 IEEE International Conference onComputational Intelligence for Measurement Systems and Applications (CIMSA 2010), pp. 18-23, September, 2010.

Pending publications11. R. Donida Labati, A. Genovese, V. Piuri, F. Scotti, "Fully Touchless On the Move 3D Fingerprint

Recognition: a Less-Constrained Two-View Approach".

Other publications12. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Wildfire Smoke Detection using

Computational Intelligence Techniques Enhanced with Synthetic Smoke Plume Generation",in IEEE Transactions on Systems, Man and Cybernetics–Part A: Systems and Humans, 2013.


Publications (4/4)

March 18, 2014

13. S. De Capitani di Vimercati, A. Genovese, G. Livraga, V. Piuri, and F. Scotti, "Privacy andSecurity in Environmental Monitoring Systems: Issues and Solutions", in Computer andInformation Security Handbook, 2nd Edition, J. Vacca (ed.), Morgan Kaufmann, 2013.

14. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "A Virtual Environment for theSimulation of 3D Wood Strands in Multiple View Systems for the Particle SizeMeasurements", in 2013 IEEE Int. Conf. on Computational Intelligence and VirtualEnvironments for Measurement Systems and Applications (CIVEMSA 2013), July 15 - 17, 2013.

15. R. Donida Labati, A. Genovese, V. Piuri, and F. Scotti, "Low-cost Volume Estimation by Two-view Acquisitions: A Computational Intelligence Approach", in 2012 International JointConference on Neural Networks (IJCNN 2012), pp. 1092-1099, June, 2012.

16. A. Genovese, R. Donida Labati, V. Piuri, and F. Scotti, "Wildfire smoke detection usingcomputational intelligence techniques", in IEEE International Conference on ComputationalIntelligence for Measurement Systems and Applications (CIMSA 2011), pp. 1-6, September,2011.

17. A. Genovese, R. Donida Labati, V. Piuri, and F. Scotti, "Virtual environment for syntheticsmoke clouds generation", in IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurement Systems (VECIMS 2011) ", pp. 1-6, September, 2011.

Angelo Genovese – Contactless and less-constrained palmprint recognition 45March 18, 2014

Thank you for your kind attention!


Accuracy of contactless palmprintsat a fixed distance

March 18, 2014

• Accuracy of palmprints captured at a fixed distance

- 26 palms, 208 samples

EER (%)FNMR @FMR= 0.05%

FNMR @FMR= 0.10%

FNMR @FMR= 0.25%

FMR@FNMR= 0.25%

FMR@FNMR= 0.50%

0.25 0.27 0.27 0.27 0.14 0.02


Experimental results:Accuracy of contactless fingerprints

March 18, 2014

• Scenario evaluation

- Contact-based and

contactless acquistions

- 1040 samples

(8 samples × 130 individuals)


Experimental results:Compatibility of contactless fingerprints

March 18, 2014

• Performed test

- 1040 contact-based samples

- 1040 unwrapped 3d samples

- EER = 1.97%

contactless and less-constrained palmprint recognition - ph.d. presentation

Engineering

contactless recognition

biometric recognition

constrained techniques

contactless method

contactless sensors

angelo genovese contactless

behavioral biometrics

accurate traits