soft-margin learning for multiple feature- kernel...
TRANSCRIPT
Soft-Margin Learning for Multiple Feature-Kernel Combinations with Domain
Adaptation, for Recognition in Surveillance Face Dataset
Samik Banerjee* & Sukhendu Das#
VPLab, CS&E, IIT Madras, Indiawww.cse.iitm.ac.in/~vplab
*[email protected]# [email protected]
CVPRW-Biometrics; June 2016
OverviewO Motivation and Problem definitionO Multiple-kernel learningO Soft-Margin Learning for Multiple feature-
kernel combination (SML-MFKC)O Domain Adaptation (DA)O Proposed TechnologyO Results and discussionO Conclusion & References
Motivation and Problem Statement
Problems:• Low Resolution• Blur• Low contrast• Aliasing effect• Poor Illumination• Different Camera Parameters
Aim:Design of an effective FaceRecognition algorithm, thatexploits DA in RKHS to obtaina feature representation withsimilar distributions ofgallery and probe, within anoptimal feature-kernelcombination by MKL.
Gallery Probes
Multiple Kernel Learning• Given M kernel functions , … , , find a positive combination
of these kernels such that the resulting kernel is « optimal » in some sense,
• Need to learn together the kernel coefficients and the SVM parameters.
• Extract features from all available sources• Construct Kernel Matrices for
Different Features Different Kernel Types Different Kernel Parameters
• How to find Optimal Kernel-Feature Combination and Kernel Classifier for each feature ?
, = , , with ≥ 0, = 1
Our aim is to transform the kernel-selection problem of MKL to a feature-selection* problem, to obtain an optimal feature-kernel combinations.*(The method is modified based on the methodology proposed by Gehler et al., ICCV, 2009).
SML-MFKC • Transformation of the kernel-selection problem of MKL to a
feature-selection problem.• Feature set:
Eigenfaces, Fisherfaces, Weberfaces, Gaborfaces, BOW, FV−SIFT, VLAD−SIFT
• Kernel Set:
Linear, Polynimial, Gaussian, RBF, Chi−square, RBF + Chi−square
• Optimal feature-kernel combination < , >; ; .• is projected to RKHS using to obtain a new feature
representation.
SML-MFKC• The cost function that gives the optimal combination of the
kernel and the feature, :sup argmin∈ℛ ∑ + ∑ ( , + ∑ )s.t. ∑ = 1 , ≥ 0, ∀ ,
– where, , = max 0,1 − ,= No. of kernels, = No. of features,= weight coefficient for the m-th feature ( ) paired with the q-th kernel.and are the parameters of SVM,is the local minima, Cis a constant.
• Block-wise gradient-descent based approach is used for solvingthe minimization problem.
• The best selected feature, is based on the supremum over the set, .
Domain Adaptation (DA)Target Domain : Probe SamplesSource Domain : Gallery Samples
• Reasons for domain adaptation– Difference in resolution– Blur– Noise– Low-contrast– Different camera parameters
• Since the features in the RKHS will also have different distributions, we perform DA based on the technique proposed by Hoffman et al., ICLR, 2013.
Domain Adaptation (DA)• The normal to the affine hyperplane associated with the -th kernel in a binary
SVM is denoted as ; = 1, … , • The offset of that hyperplane from the origin is .• is the transformation of the source hyperplane parameters, .• Training points & labels in the source domain:{< , >,… ,< , >}• Target points & labels in the target domain:{< , >,… ,< , >}• Hinge Loss: ℒ , , = max(0,1 − ( , ) · )• The cost function: , , =+∑ + ∑ ℒ , . 1 , + C ∑ ℒ , 1 ,• penalizes the source classification error.• penalizes the target adaptation error.• The minimization of using coordinate descent approach yields the
transformation matrix .
Proposed framework of DA:
Proposed Methodology• Overall Framework
Datasets
FR_SURV 1.75 1.75
SCFace 1.70 1.70
ChokePoint 1.20 1.25Gallery Degraded
GalleryEnhanced Probe
Probe
FR_SURV
SCFace
ChokePoint
(From Dataset)
(From Dataset)
Training Phase
Dataset No. of Probe Samples for DA
FR_SURV 5 per subject,from 20 random samples
SCFace 3 per subject,from 30 random samples
ChokePoint 6 per subject,from 5 males & 2 females per profile
DatabasesFR_SURV SCFace ChokePoint
# Subjects 51 130 54# Surveillance Cameras 1 5 3
Gallery Image size 150 x 150 250 X 250 80 X 80Probe Image Size 33 X 33 15-45 X 15-45 80 X 80
Surveillance Scenario Outdoor Indoor Indoor
Gallery
Probe
D/B Name FR_SURV SCFace ChokePoint
• Rank-1 Recognition Rate (%)
Experimental Results
Sl. Algorithm SCFace FR_SURV ChokePoint1 EDA1 [14] 47.65 7.82 54.212 COMP_DEG [11] 4.32 43.14 62.593 MDS [13] 42.26 12.06 52.134 KDA1 [14] 35.04 38.24 56.255 Gopalan [10] 2.06 2.06 58.626 Kliep [12] 37.51 28.79 63.287 Deep Face [15]* 41.25 29.35 62.158 Naïve 77.45 48.23 69.519 BaseMKL (only VLAD-SIFT) 53.36 36.54 66.1210 Proposed 79.86 56.44 85.59
* - Omkar M. Parkhi, Andrea Vedaldi and Andrew Zisserman; Deep Face Recognition; in Proceedings of the British Machine Vision Conference (BMVC), pages 41.1-41.12. BMVA Press, September 2015.
Comparison with several DL methods, on SURVEILLANCE Face Datasets
No. Method # CNN LayersRank-1 Recognition Rate
FR_SURV SCFace Choke Point
1 FV Faces + AlexNet 8 12.64 35.24 61.592 DeepFace [15] 19 29.35 41.25 62.154 DeepID-2,2+,3 60 34.94 32.92 66.86
5 FaceNet + Alignment [17] 22 36.53 48.21 71.65
6 VGG Face Descriptor + Deep Face [16] 16 32.57 46.25 69.86
7 Proposed Method - 56.44 79.86 85.59
RANK-1 RECOGNITION RATE
RANK-5 RECOGNITION RATE
RANK-10 RECOGNITION RATE
0
20
40
60
80
100
EDA1 COMP_DEG MDS KDA1 Gopalan Kliep Deep Face Naïve BaseMKL Proposed
FR_SURV
SCFace
ChokePoint
0
20
40
60
80
100
EDA1 COMP_DEG MDS KDA1 Gopalan Kliep Deep Face Naïve BaseMKL Proposed
FR_SURV
SCFace
ChokePoint
0
20
40
60
80
100
EDA1 COMP_DEG MDS KDA1 Gopalan Kliep Deep Face Naïve BaseMKL Proposed
FR_SURV
SCFace
ChokePoint
• ROC Plots
• CMC Plots
Experimental Results
FR_SURV SCFace ChokePoint
• An efficient method to tackle the problem of low-contrastand low-resolution proposed for near-frontal surveillanceface images.
• Novel method using SML-MFKC to obtain an optimal pairingof feature and kernel, followed by DA in RKHS.
• Superiority of performance, shown using ROC, CMC andRank-1 Recognition Rate than the other techniques, usingthree real-world surveillance face datasets.
• Deep learning fails to perform well for FR undersurveillance scenarios.
• Larger set of subjects and performance with off-frontalfaces may be studied, as future scope of work
Conclusions
References1. A. Asthana, S. Zafeiriou, S. Cheng, and M. Pantic. Incremental face alignment in the wild. In CVPR,
2014.2. J. Goldberger, S. Gordon, and H. Greenspan. An efficient image similarity measure based on
approximations of kl-divergence between two Gaussian mixtures. In ICCV, IEEE, 2003.3. M. Grgic, K. Delac, and S. Grgic. SCface - surveillance cameras face database. Multimedia tools and
applications, 2011.4. Y. Wong, S. Chen, S. Mau, C. Sanderson, and B. C. Lovell. Patch-based probabilistic image quality
assessment for face selection and improved video-based face recognition. In IEEE BiometricsWorkshop, Computer Vision and Pattern Recognition (CVPR) Workshops, pages 81-88. IEEE, June2011.
5. Ahonen, Timo, Abdenour Hadid, and Matti Pietikainen. "Face description with local binarypatterns: Application to face recognition." Pattern Analysis and Machine Intelligence, IEEE, 28.12(2006): 2037-2041.
6. F. R. Bach, G. R. Lanckriet, and M. I. Jordan. Multiple kernel learning, conic duality, and the smoalgorithm. In Proceedings of the 21st International Conference on Machine learning, page 6. ACM,2004.
7. P. Gehler and S. Nowozin. On feature combination for multiclass object classification. In ICCV. IEEE,2009.
8. J. Hoffman, E. Rodner, J. Donahue, T. Darrell, and K. Saenko. Efficient learning of domain-invariantimage representations. ICLR, 2013.
9. Juefei-Xu, Felix, and Marios Savvides. "Single face image super-resolution via solo dictionarylearning." In Image Processing (ICIP), 2015. IEEE, 2015.
References10. R. Gopalan, R. Li, and R. Chellappa. Domain adaptation for object recognition: An
unsupervised approach. In ICCV, 2011.11. S. Rudrani and S. Das. Face recognition on low quality surveillance images, by
compensating degradation. In ICIAR, LNCS, Springer, 2011.12. M. Sugiyama, S. Nakajima, H. Kashima, P. Von Bunau, and M. Kawanabe. Direct
importance estimation with model selection and its application to covariate shiftadaptation. In NIPS, 2007.
13. S. Biswas, K. W. Bowyer, and P. J. Flynn. Multidimensional scaling for matching low-resolution face images. IEEE Transactions on Pattern Analysis and Machine Intelligence,2012.
14. S. Banerjee, S. Samanta, and S. Das. Face recognition in surveillance conditions withbag-of-words, using unsupervised domain adaptation. In ICVGIP. ACM, 2014.
15. Parkhi Omkar M., Andrea Vedaldi, and Andrew Zisserman. "Deep facerecognition." Proceedings of the British Machine Vision 1.3 : 6, 2015.
16. Sun, Yi, Xiaogang Wang, and Xiaoou Tang. "Deep learning face representation frompredicting 10,000 classes." In Proceedings of the IEEE Conference on Computer Visionand Pattern Recognition, pp. 1891-1898. 2014.
17. Schroff, Florian, Dmitry Kalenichenko, and James Philbin. "Facenet: A unifiedembedding for face recognition and clustering." In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition, pp. 815-823. 2015.
