Strategies for improving face recognition from videoDeborah Thomas, Nitesh V. Chawla, Kevin W. Bowyer, and Patrick J. Flynn

Computer Vision Research Lab, University of Notre Dame (http://www.nd.edu/~cvrl)

Goals Dataset

• Notre Dame dataset:

• 105 subjects – gallery: indoor, probe: outdoor

• 3 cameras: HD JVC, DV Canon, iSight Webcam

• Honda/UCSD dataset: 20 subjects, 1- 4 clips each

• Hypothesis: The Mahalanobis cosine distance between images in PCA space reflects their difference

• Process:

• Project images from subject into PCA space

• Pick image with largest total Mahalanobis cosine distance from all others

• Pick successive images that are farthest away from the previously selected image

Using Mahalanobis cosine distances in PCA space to determine diversity

Results: UCSD/Honda dataset

Combining quality measure with difference

Acknowledgements: Biometrics research at the University of Notre Dame is supported bythe National Science Foundation under grant CNS 0130839, by theCentral Intelligence Agency, by the National Geo-Spatial IntelligenceAgency, by UNISYS Corp., and by the US Department of Justice.

Using K-means clustering to group similar images

• Improve performance of face recognition from video

•Exploit multiple frames available in a given clip

•Select a minimal set of frames to represent the subject

• Hypothesis: Image clusters in PCA space represent images that are similar to each other

• Process:

• Project images from subject into PCA space

• Use retained dimensions to cluster images

• One image per cluster for N-frame representation

• FaceIt’s faceness measure: Confidence that image contains a face• Three approaches:

• LAD: Use Mahalanobis cosine distance to determine distance, use N frames most different from each other• LADHF: Project top 35 frames with highest faceness into PCA space, use N frames most different from each other• CLS: Create K clusters of images and pick image with highest faceness from N clusters for representation

• Experiments: • Use up to 20 frames• Compare to choosing images that are equally spaced in time

Results –Canon as probe and gallery

• Lee et al. • Use clusters in PCA space to determine pose• Posterior probabilities: Identity of current frame conditioned on previous frames• Rank One recognition rate:

• 2003: 92.1 %• 2005: 98.9 %

• Our approach: • Using 7 frames: 98.8 %

Example images – Notre Dame Dataset

Gal

lery

: O

utd

oo

rP

rob

e:

Ind

oo

r

DV Canon HD JVC iSight Webcam

Rank One Recognition Rate

0

20

40

60

80

100

120

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35

Number of frames

Ran

k O

ne

Rec

og

nit

ion

Rat

e

Kriegman, 2003

Kriegman, 2005

NEHF

Rank One Recognition Rate

0

10

20

30

40

50

60

70

80

90

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Number of frames

Ran

k o

ne

Rec

og

nit

ion

Rat

e

LAD

LADHF

CLS

EST