machine learning for person identification
TRANSCRIPT
![Page 1: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/1.jpg)
Machine Learning forPerson Identification
Wei-Shi Zheng (郑伟诗)
机器智能与先进计算教育部重点实验室
![Page 2: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/2.jpg)
2
Brief Introduction of ML for Biometrics ML for Person Re-identification
Distance Metric Learning View Change Invariant Features Partial Re-id Low Resolution Video-based Re-id Cross Scenario Transfer Open-world Modelling Depth Re-identification
Summary
Outline
![Page 3: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/3.jpg)
A BRIEF INTRODUCTION ONMACHINE LEARNING FORPERSON IDENTIFICATION
![Page 4: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/4.jpg)
4
Biometrics
![Page 5: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/5.jpg)
5
Why Machine Learning is Needed
![Page 6: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/6.jpg)
6
Why Machine Learning is Needed
Smallsamplesize
Large-scalesamplesize
![Page 7: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/7.jpg)
7
weakly supervised penalty
Preprocessing Propose a two-step framework Propose a weakly supervised penalty: guide the learning
- J. T. Kwok and I. W. Tsang, “The pre-image problem in kernel methods,” IEEE Trans. Neural Netw., vol. 15, no. 6, pp. 1517–1525, Nov. 2004.- Wei-Shi Zheng, JianHuang Lai, and Pong C. Yuen, "Penalized Pre-image Learning in Kernel Principal Component Analysis," IEEE Trans. on Neural Networks, vol. 21, no. 4, pp. 551-570, 2010.
![Page 8: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/8.jpg)
8
Super-resolution Sparse Coding
Jianchao Yang et al. Image Super-Resolution Via Sparse Representation. IEEE Trans. on Image Processing, 2010.
![Page 9: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/9.jpg)
9
Super-resolution Deep Processing
Chao Dong, Chen Change Loy, Kaiming He, Xiaoou Tang. Image Super-Resolution Using Deep Convolutional Networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015.
Patch Extraction and Representation
Non-Linear Mapping
Reconstruction
![Page 10: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/10.jpg)
10
Alignment PCA Alignment
Weihong Deng, Jiani Hu, Jiwen Lu, Jun Guo. Transform-Invariant PCA: A Unified Approach to Fully Automatic FaceAlignment, Representation, and Recognition. IEEE TPAMI, 2014.
![Page 11: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/11.jpg)
11
Feature Extraction Subspace Learning
WWSS bw1
Class 1
Class 2
Between-class covariance matrix
Within-class covariance matrix
P.N. Belhumeur, J. Hespanha, D.J. Kriegman, Eigenfaces vs. Fisherfaces: recognition using class specific linear projection, IEEE Trans. Pattern Anal. Mach. Intell. 19 (7) (1997) 711–720.
Fisherface
![Page 12: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/12.jpg)
12
Feature Extraction LFDA
S. Pedagadi, J. Orwell, S. Velastin and B. Boghossian, Local Fisher Discriminant Analysis for Pedestrian Re-identification, 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013, pp. 3318-3325.
Distribution of pedestrian features is multi-modal.
![Page 13: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/13.jpg)
13
Feature Extraction Subspace Learning
reshape1
2
3
4
5
6
7
8
9
4
6
123
5
789
4
6
123
5
789
1
2
3
4
5
6
7
8
9
Traditional ApproachTwo-dimensional Approach
Using Geometric Informationwl
wr
w
2D-LDA could lose the cross-covariance information between rows or columns
Wei-Shi Zheng, JianHuang Lai, and Stan Z. Li, "1D-LDA versus 2D-LDA: When Is Vector-based Linear Discriminant Analysis Better than Matrix-based?" Pattern Recognition, vol. 41, no. 7, pp. 2156-2172, 2008.
![Page 14: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/14.jpg)
14
Feature Extraction Non-negativity Matrix Factorization
D.D. Lee, H.S. Seung. Learning the parts of objects by non-negative matrix factorization, Nature, 1999.
![Page 15: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/15.jpg)
15
Feature Extraction Pixel Dispersion Penalty
Wei-Shi Zheng, JianHuang Lai, Shengcai Liao, and Ran He, "Extracting Non-negative Basis Images Using Pixel Dispersion Penalty," Pattern Recognition, vol. 45, no. 8, pp. 2912-2926, 2012.
Pixel Dispersion Penalty
![Page 16: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/16.jpg)
16
Feature Extraction Binary Coding
Jiwen Lu et al., Learning Compact Binary Face Descriptor for Face Recognition. IEEE TPAMI, 2015.
![Page 17: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/17.jpg)
17
Feature Selection Ensemble of localized features
Ensemble of localized features exploit color histograms and textures.
D. Gray, H. Tao. Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features. European Conference on Computer Vision (ECCV), 2008
Color histograms Textures
![Page 18: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/18.jpg)
18
Feature Selection Floatboost
Stan Li et al. FloatBoost Learning and Statistical Face Detection. IEEE TPAMI, 2004.
![Page 19: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/19.jpg)
19
Dimension Reduction Subspace Learning
Turk, Matthew A and Pentland, Alex P. Face recognition using eigenfaces. IEEE CVPR, 1991.
![Page 20: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/20.jpg)
20
Classification Sparse Representation-based Classifier
- J. Wright, A.Y. Yang, A. Ganesh, S.S. Sastry, and Y. Ma, “Robust Face Recognition via Sparse Representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210-227, Feb. 2009.
- Ran He, Wei-Shi Zheng, and BaoGang Hu. Maximum Correntropy Criterion for Robust Face Recognition.IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 33, no. 8, pp. 1561 - 1576, 2011.
SRC:J. Wright et al.
Non-negativity:He et al.
![Page 21: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/21.jpg)
21
Classification Distance Metric Learning
Pulling positive pairs by minimizing intra-class distances
Pushing negative pairs by enlarging inter-class distances
Similarity is measured by Mahalanobis distance
Weinberger, Kilian Q., and Lawrence K. Saul. "Distance metric learning for large margin nearest neighbor classification." Journal of Machine Learning Research, 2009
![Page 22: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/22.jpg)
22
Online Learning Incremental Learning
1……
2 3 4 5 6 7
class1 class2 class3 ……Real World
Data Stream
…
InitializeModel/Classifier
UpdateModel/Classifier
… UpdateModel/Classifier
Samples obtained in a chunk way
![Page 23: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/23.jpg)
23
Online Learning One-pass Learning
Zhaoze Zhou, Wei-Shi Zheng, et al. One-pass online learning: A local approach. Pattern Recognition, 2016.
![Page 24: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/24.jpg)
24
Online Learning One-pass Learning
Zhaoze Zhou, Wei-Shi Zheng, et al. One-pass online learning: A local approach. Pattern Recognition, 2016.
![Page 25: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/25.jpg)
25
Online Learning One-pass Learning
Zhaoze Zhou, Wei-Shi Zheng, et al. One-pass online learning: A local approach. Pattern Recognition, 2016.
![Page 26: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/26.jpg)
26
Deep Learning DeepID
Yi Sun et al. Deep learning face representation from predicting 10,000 classes. In CVPR, 2014
![Page 27: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/27.jpg)
27
Deep Feature Deep Re-id
W. Li, R. Zhao, T. Xiao and X. Wang, "DeepReID: Deep Filter Pairing Neural Network for Person Re-identification," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
The filter pairing neural network (FPNN) jointly handles misalignment, photometric and geometric transforms, occlusions and background clutter.
![Page 28: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/28.jpg)
28
Deep Feature Deep Learning with Domain Guided Dropout
T. Xiao, H. Li, W. Ouyang and X. Wang. Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification. IEEE International Conference on Computer Vision, 2016
Learning deep feature representations from multiple domains with Convolutional Neural Networks (CNNs).
![Page 29: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/29.jpg)
29
Deep Learning Deep RE-ID+Mirror
Shangxuan Wu, Ying-Cong Chen, Xiang Li, An-Cong Wu, Jin-Jie You, and Wei-Shi Zheng*. An Enhanced Deep Feature Representation for Person Re-identification. WACV, 2016.
Yingcong Chen, Wei-Shi Zheng*, and Jian-Huang Lai"Mirror Representation for Modeling View-specific Transform in Person Re-identification," IJCAI, 2015.
Cross-view metric, IJCAI 2015
![Page 30: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/30.jpg)
30
Cross Modality Learning
L. Lin; G. Wang; W. Zuo; F. Xiangchu; L. Zhang, "Cross-Domain Visual Matching via Generalized Similarity Measure and Feature Learning," in IEEE Transactions on Pattern Analysis and Machine Intelligence , 2016
Learning deep feature representations for two modalities with domain-specific and shared sub-networks and a generalized similarity measure.
Generalized-Similarity-based Feature Learning
![Page 31: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/31.jpg)
31
Cross Modality Learning
Existing VIS based face recognition system
Labelled pair-wise VIS-NIR training samples
Newly captured unlabelled NIR images
(1) Assume a set of VIS-NIR pairs of training people is available(2) Guide the learned VIS-NIR matching upon training to facilitate the matching for target
ones.
Jun-Yong Zhu (student), Wei-Shi Zheng*, Jian-Huang Lai, Stan Z. Li. Matching NIR Face to VIS Face using Transduction. IEEE Transactions on Information Forensics and Security, vol. 9, no. 3, pp. 501-514, 2014.
![Page 32: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/32.jpg)
32
TOO A LOT Manifold Learning Subspace: ICA, CCA Dictionary Learning Semi-supervised Learning Other Classifiers:
Bayes, Adaboost, Random Forest Active Learning Unsupervised Discriminant Learning
![Page 33: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/33.jpg)
MACHINE LEARNING FORPERSON RE-IDENTIFICATION
![Page 34: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/34.jpg)
34
Brief Introduction of ML for Biometrics ML for Person Re-identification
Distance Metric Learning View Change Invariant Features Partial Re-id Low Resolution Video-based Re-id Cross Scenario Transfer Open-world Modelling Depth Re-identification
Summary
Outline
![Page 35: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/35.jpg)
35
Background: Visual Surveillance
![Page 36: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/36.jpg)
36
Person Re-identification
Detecting target objects (Cars, pedestrian, bags etc.)
Matching, Tracking
What is he
doing?
Camera Network Understanding
![Page 37: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/37.jpg)
37
PersonRe-identification
Face ImageComputing
Activity
Person Re-identification Concern the person who is joining an activity
Tracking him/her across camera-views Identifying him/her when we can capture his/her
face very well
Recognising/Searching face images in a Large Dataset
![Page 38: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/38.jpg)
38
A key component to track people across disjoint views
Suspect, Bomb in Boston, USA(2013)
Suspect, Terrorist Attack, Kunming, China (2014)
Person Re-identification
![Page 39: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/39.jpg)
39
kidnapping
Person Re-identification
![Page 40: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/40.jpg)
40
FeatureExtraction
Distance Learning
PeopleDetection
Person Re-identification
![Page 41: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/41.jpg)
41
Person Re-identification: Challenges Main Variations
View Lighting Occlusion Low Resolution Cloth Change
![Page 42: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/42.jpg)
How to measure the differences between two person images
Wei-Shi Zheng et al. Re-identification by Relative Distance Comparison. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI). 2013.
![Page 43: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/43.jpg)
43
Our Idea
difference
f
f
bipartite rankingpositive negative
data
Triple based Learning: Bipartite Ranking
![Page 44: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/44.jpg)
44
Intra-class Distance < Inter-class Distance
positive differencevector
negative differencevector
OBJECTIVE
Triple based Learning: Bipartite Ranking A Relative Distance Comparison Model
Reduce the sensitivity for comparison Enhance the performance ( 20~30%,i-LIDS, VIPeR)
soft margin measure
Wei-Shi Zheng et al. Re-identification by Relative Distance Comparison. IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI). 2013.
![Page 45: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/45.jpg)
45
Entry‐wise Absolute Difference Vector
Relative Distance Learning can be more robust in the absolute distance space
Triple based Learning: Bipartite Ranking
![Page 46: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/46.jpg)
46
Learn the projection vectors each by each
Triple based Learning: Bipartite Ranking
![Page 47: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/47.jpg)
Convergence
Triple based Learning: Bipartite Ranking
![Page 48: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/48.jpg)
48
Entry‐wise Absolute Difference Vector
Relative Distance Learning can be more robust in the absolute distance space
Triple based Learning: Bipartite Ranking
![Page 49: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/49.jpg)
49
Ensemble RDC: Motivation• RDC: Large space complexity
• RDC: Trapped in locally optimal solutionEnsemble RDC: Modelling
• Randomly dividing the set into small groups• Learning a set of weak RDC models• Boosting them
Ensemble Metric Learning
Triple based Learning: Bipartite Ranking
![Page 50: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/50.jpg)
5019
Re‐identification (i‐LIDS&VIPeR)
Triple based Learning: Bipartite Ranking
![Page 51: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/51.jpg)
51
XQDA Local Maximal Occurrence Representation (CVPR2015)
Shengcai Liao, Yang Hu, Xiangyu Zhu, and Stan Z. Li. Person Re-identification by Local Maximal Occurrence Representation and Metric Learning. CVPR, 2015.
An effective handcrafted feature and a distance metric are proposed.
![Page 52: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/52.jpg)
52
Deep Distance Improved Deep Learning Architecture (CVPR2014)
E. Ahmed, M. Jones and T. K. Marks, "An improved deep learning architecture for person re-identification," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014
A network for simultaneously learning features and a corresponding similarity metric for person re-identification.
![Page 53: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/53.jpg)
53
Deep Distance
Nonlinear Local Metric Learning for Person Re-identification
By Siyuan Huang, Jiwen Lu, et al.
![Page 54: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/54.jpg)
54
Deep Feature Deep Learning with Domain Guided Dropout (CVPR2016)
T. Xiao, H. Li, W. Ouyang and X. Wang, "Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification,“ IEEE International Conference on Computer Vision (CVPR), 2016
Learning deep feature representations from multiple domains with Convolutional Neural Networks (CNNs).
![Page 55: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/55.jpg)
55
Learning-based Mid-level Feature Deep Attribute Learning (ECCV2016)
Chi Su, Shiliang Zhang, Junliang Xing, Wen Gao, Qi Tian, "Deep Attributes Driven Multi-Camera Person Re-identification“, European Conference on Computer Vision (ECCV), 2016.
![Page 56: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/56.jpg)
56
Learning-based Mid-level Feature Salience Learning for Re-ID (CVPR2013)
R. Zhao, W. Ouyang and X. Wang, "Unsupervised Salience Learning for Person Re-identification," Computer Vision and Pattern Recognition (CVPR), 2013
Small salient regions are exploited to match persons.
![Page 57: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/57.jpg)
Person Re-Identification by Unsupervised L1 Graph Learning
Unsupervised Learning
The unsupervised Re-ID problem is formulated by graph regularized dictionary learning method.
E. Kodirov, T. Xiang, Z. Fu, S. Gong, “Person Re-Identification by Unsupervised L1 Graph Learning”, ECCV, 2016
![Page 58: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/58.jpg)
58
Post-rank Search Re-ranking Re-ID (ICCV2013)
C. Liu, C. C. Loy, S. Gong and G. Wang, "POP: Person Re-identification Post-rank Optimisation," IEEE International Conference on Computer Vision (ICCV), 2013
Strong negatives are labeled by human operator in the re-ranking process.
![Page 59: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/59.jpg)
59
What is Wrong with Current Metrics• The view label Information is not explicitly used
• The distributions of person images across camera views are different
• Existing metrics are learned for each scenario and cannot generalize very well
![Page 60: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/60.jpg)
When View Labels are available, how to model the view transform more accurately
Yingcong Chen, Wei-Shi Zheng*, and Jian-Huang Lai, "Mirror Representation for Modeling View-specific Transform in Person Re-identification," International Joint Conference on Artificial Intelligence (IJCAI), 2015.
Ying-Cong Chen, Wei-Shi Zheng*, Jianhuang Lai, Pong C. Yuen. An Asymmetric Distance Model for Cross-view Feature Mapping in Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, 2016.
![Page 61: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/61.jpg)
61
• Illumination, viewpoint or camera features vary across views, and distributions of each view are different.
• View‐Specific Mappings can be adopted to correct differentdistributions of views.
Usefulness of View Label Information
Mirror Representation
![Page 62: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/62.jpg)
62
Mirror Representation
Zero-Padding Augmentation
![Page 63: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/63.jpg)
63
Illustration of Zero‐Padding Augmentation
Mirror Representation
![Page 64: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/64.jpg)
64
Limitation of Zero-Padding
![Page 65: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/65.jpg)
65
Reformulation of Zero-Padding
generalise
![Page 66: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/66.jpg)
66
r0 1
A Feature-Level Discrepancy Modeling
![Page 67: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/67.jpg)
67
Is it Not Optimal?
A Transformation-Level Discrepancy Modeling
![Page 68: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/68.jpg)
68
A Transformation-Level Discrepancy Modeling
![Page 69: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/69.jpg)
69
can be solved by traditional metric learning (with ridge regularization)
Mirror Representation
A Transformation-Level Discrepancy Modeling
![Page 70: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/70.jpg)
70
The best is marked red, and the second best is marked blue.
Effectiveness of Mirror Representation
![Page 71: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/71.jpg)
71
Performance
![Page 72: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/72.jpg)
72
Shangxuan Wu, Ying-Cong Chen, Xiang Li, An-Cong Wu, Jin-Jie You, and Wei-Shi Zheng*. An Enhanced Deep Feature Representation for Person Re-identification. WACV 2016.
Yingcong Chen (student), Wei-Shi Zheng*, and Jian-Huang Lai"Mirror Representation for Modeling View-specific Transform in Person Re-identification," IJCAI, 2015.
Cross-view metric, IJCAI 2015
Deep RE-ID+Mirror
![Page 73: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/73.jpg)
73
Deep RE-ID+Mirror
![Page 74: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/74.jpg)
Partial Re‐identification
Wei-Shi Zheng, Xiang Li, Tao Xiang, Shengcai Liao, JianHuang Lai, Shaogang Gong. Partial Person Re-identification. In IEEE Conf. on Computer Vision (ICCV), 2015 (oral)
![Page 75: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/75.jpg)
75
Partial Observation
![Page 76: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/76.jpg)
76
Matching Fusion
Annotating Partial Part by Operator
or Detecting it automatically
Local-to-local Matching
Global-to-local Matching
Partial Re-ID
Wei-Shi Zheng, Xiang Li, Tao Xiang, ShengcaiLiao, JianHuang Lai, Shaogang Gong. Partial Person Re-identification. ICCV, 2015.
![Page 77: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/77.jpg)
77
Local-to-local Matching— Ambiguity-Sensitive Matching classifier (AMC)
Constructing patch level descriptors from gallery person images to form a dictionarywhere
probe patch feature
where
Ambiguity Score
Ambiguity constraint
Sparsity constraint
Classifying a probe partial image :
Partial Re-ID
![Page 78: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/78.jpg)
78
Example of AMC used for partial person matching
Partial Re-ID
![Page 79: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/79.jpg)
79
Global-to-local Matching— Sliding Window Matching (SWM)
Fusion Matching— AMC-SWM
Set up a sliding window of the same size as the probe image.
SMM distance
AMC distance
Use L1-norm to measure the distance.
Search for the most similar image region within each gallery image.
Partial Re-ID
![Page 80: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/80.jpg)
80
A New Partial REID dataset (new collection, released now: http://isee.sysu.edu.cn/resource):
600 images of 60 people
5 full-body images and 5 partial images per person
Fig. Examples of partial person images (first row), and the input partial part annotated by an operator for recognition (second row) and the corresponding full-body images (third row).
Partial Re-ID
![Page 81: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/81.jpg)
81
Two Simulated datasets: P-iLIDs and P-CAVIAR Based on i-LIDS (476 images of 119 people) & CAVIAR (1220 images of 72 people).
Randomly selected half of all the images of each person and replaced them with the partial images.
Fig. Examples of partial person images (firstrow) and the corresponding full images (second row). From left to right, columns 1–3 are from P-i-LIDS, and columns 4–6 from P-CAVIAR.
Partial Re-ID
![Page 82: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/82.jpg)
82
Results on Partial REID dataset The test sets were randomly selected using 70% of the individuals. Both single-shot and multi-shot experiments were conducted.
Partial Re-ID
![Page 83: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/83.jpg)
83
Results on two simulated datasetsPartial Re-ID
![Page 84: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/84.jpg)
84
Evaluation of the two matching components
Parameter Evaluation
Partial Re-ID
![Page 85: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/85.jpg)
85
Illustration of Matching Examples on Partial REID dataset
Partial Re-ID
![Page 86: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/86.jpg)
Re‐identification under Low Resolution
![Page 87: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/87.jpg)
87
Our Proposed Multi-scale Learning Model
JUDEA : joint multi-scale discriminant component analysisXiang Li, Wei-Shi Zheng*, Xiaojuan Wang, Tao Xiang, Shaogang Gong. Multi-scale Learning for Low-resolution Person Re-identification. IEEE Conf. on Computer Vision (ICCV), 2015.
Low Resolution
![Page 88: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/88.jpg)
88
Or Proposed Multi-scale Learning Model
Low Resolution
Cross-scale Image Domain Alignment
![Page 89: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/89.jpg)
89
Low Resolution
![Page 90: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/90.jpg)
Video‐based Re‐identification
Jinjie You, Ancong Wu, Xiang Li, Wei-Shi Zheng*. Top-push Video-based Person Re-identification. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016.
![Page 91: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/91.jpg)
91
TOP-PUSH Distance Metric Learning
Jinjie You, Ancong Wu, Xiang Li, Wei-Shi Zheng*. Top-push Video-based Person Re-identification. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2016.
/
The goal is to learn a Mahalanobis metric:
A triplet hingeloss function:
![Page 92: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/92.jpg)
92
Top-push Distance Metric Model
Top-push constraint
TOP-PUSH Distance Metric Learning
![Page 93: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/93.jpg)
93
PRID 2011
TOP-PUSH Distance Metric Learning
![Page 94: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/94.jpg)
94
iLIDS-VID (Extracted from the i-LIDS dataset )
TOP-PUSH Distance Metric Learning
![Page 95: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/95.jpg)
95
TOP-PUSH Distance Metric Learning
![Page 96: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/96.jpg)
96
Video-based Re-ID Discriminative Video fragments selection and Ranking (ECCV2014)
T. Wang, S. Gong, X. Zhu and S. Wang, "Person Re-Identication by Video Ranking,“ European Conference on Computer Vision (ECCV), 2014
The video-based model automatically selects the most discriminative video fragments and learns a ranking function simultaneously.
![Page 97: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/97.jpg)
97
Video RE-ID
Niall McLaughlin, Jesus Martinez del Rincon, Paul Miller. RecurrentConvolutional Network for Video-based Person Re-Identification.CVPR 2016
![Page 98: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/98.jpg)
Gallery
Probe
Labelling images across camera views is costly
Person Re-identification: Labelling
![Page 99: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/99.jpg)
Labeling images is costly and even prohibitive in some scenarios
![Page 100: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/100.jpg)
Is it possible to use collected images in other scenarios to boost the learning in the target scenario?
Cross‐scenario Transfer Person Re‐identification
Xiaojuan Wang, Wei-Shi Zheng*, Xiang Li, and Jianguo Zhang. Cross-scenario Transfer Person Re-identification. IEEE Transactions on Circuits and Systems for Video Technology, DOI: 10.1109/TCSVT.2015.2450331, 2015.
![Page 101: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/101.jpg)
101
An AsymmetricMulti-task Modelling
Framework
![Page 102: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/102.jpg)
102
shared latent subspace:
source task-specific subspace:
target task-specific subspace:
Cross-scenario Transfer ModelingTransfer one source dataset
Transfer multiple source datasets
![Page 103: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/103.jpg)
103
The projection of a target sample
The projection of a source sample
Cross-scenario Transfer ModelingTransfer one source dataset
Transfer multiple source datasets
![Page 104: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/104.jpg)
104
To maximize local inter‐class variances and meanwhile to minimize the local intra‐class variances in both task
non‐convex
relaxation
Modeling
Cross-scenario Transfer ModelingTransfer one source dataset
Transfer multiple source datasets
![Page 105: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/105.jpg)
105
Insight
adding those measures together gives us a stronger cue on overall
discriminativeness
Cross-scenario Transfer ModelingTransfer one source dataset
Transfer multiple source datasets
![Page 106: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/106.jpg)
106
Optimization
Eq.(2) is equal to
generalized eignenvalue problem, global solution guaranteed.
Cross-scenario Transfer ModelingTransfer one source dataset
Transfer multiple source datasets
![Page 107: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/107.jpg)
107
task‐specific projection for each source dataset: by redefining:
solution could be obtained by Eq. (4)
Cross-scenario Transfer ModelingTransfer one source dataset
Transfer multiple source datasets
![Page 108: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/108.jpg)
108
instance i from task k
instance j from task l
separate data from different tasks
Constrained Asymmetric Multi-task Component Analysis
In the shared latent space, different classes from different tasks could collapse together.
![Page 109: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/109.jpg)
109
Constrained Asymmetric Multi-task Component Analysis
![Page 110: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/110.jpg)
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 111: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/111.jpg)
111
Default parameter setting: β = 0.1, γ = 0.8, α = 1 ‐ β
VIPeR
i‐LIDS
3DPeS
CAVIAR
VIPeR
3DPeS
i‐LIDS
CAVIAR
Single transfer : 12 casesMultiple transfer: 16 cases
Feature representation: concatenated color (RGB, YCbCr, HS), HoG, LBP features extracted from sub‐blocks of images
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 112: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/112.jpg)
112
Compared methods: LFDA (Pedagadi et al.), LMNN (Weinberger et al.), KISSME(Kostinger et al.), LADF (Li et al.), PCCA (Mignon et al.), RDC (Zheng et al.)
trained in three ways
using target data only (e.g. LFDA_T)
using source data only (e.g. LFDA_S)
using a pooled set of source data and target data (e.g. LFDA‐Mix)
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 113: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/113.jpg)
113
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 114: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/114.jpg)
114
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 115: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/115.jpg)
115
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 116: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/116.jpg)
116
Two observations:
• Only using source dataset for the chosen metric learning algorithms often results in better performance than only using limited target data (except for the case with VIPeR as target dataset).
• Using the pooled set of source and target data for the chosen metric learning methods almost performs almost the same as using only source data and sometimes even worse.
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 117: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/117.jpg)
117
Compared methods: TCA (Pan et al.), TFLDA (Si et al.), MT‐LMNN (Parameswaran et al.), GPLMNN (Yang et al,)
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 118: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/118.jpg)
118
With CTDD vs. Without CTDD
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 119: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/119.jpg)
119
Increase number of target training samples
Compared methodsTransfer settingExperimentDatasets
Further evaluation of cAMT‐DCA
single‐task methods
multi‐task + domain adaptation methods
![Page 120: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/120.jpg)
120
Unsupervised Cross-Dataset Transfer Learning for Person Re-identification
P. Peng, T. Xiang et al., “Unsupervised Cross-Dataset Transfer Learning for Person Re-identification”, CVPR, 2016
Transfer Learning
A Dictionary-learning-based model for cross-dataset transfer.
![Page 121: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/121.jpg)
In real world, there are quite a lot of imposters, and only a few guys are target to track
Wei-Shi Zheng, Shaogang Gong, and Tao Xiang. Towards Open-World Person Re-Identification by One-Shot Group-based Verification. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 38, no. 3, pp. 591-606, 2016.
![Page 122: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/122.jpg)
122
One-Shot Open-World Group-based Re-id
Motivation
Open-world person re-identification setting
1) A large amount of non-target imposters captured along with the target people on the watch list.
2) Their images will also appear in the probe set and some of them will look visually similar to the target people
![Page 123: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/123.jpg)
123
One-Shot Open-World Group-based Re-id
Knowledge to transferEnrich intra-class variationApproximate target intra-inter class pair(magenta line and green line)
![Page 124: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/124.jpg)
124
One-Shot Open-World Group-based Re-id
Knowledge to transfer
Enrich inter-class variationTarget specific non-target intra-inter class pair(magenta line and yellow line)
![Page 125: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/125.jpg)
125
One-Shot Open-World Group-based Re-id
Knowledge to transfer
Enrich group separationGroup separation intra-inter class pair(green line and grey line)
![Page 126: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/126.jpg)
126
One-Shot Open-World Group-based Re-id
Knowledge to transferEnrich intra-class variationApproximate target intra-inter class pair(magenta line and green line)
Enrich inter-class variationTarget specific non-target intra-inter class pair(magenta line and yellow line)
Enrich group separationGroup separation intra-inter class pair(green line and grey line)
![Page 127: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/127.jpg)
127
One-Shot Open-World Group-based Re-id
Criterion Enrich intra-class variationApproximate target intra-inter class pair(magenta line and green line)
Enrich group separationGroup separation intra-inter class pair(green line and grey line)
Enrich inter-class variationTarget specific non-target intra-inter class pair(magenta line and yellow line)
![Page 128: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/128.jpg)
128
CriterionSimilar source person image
Source intra-class
Target inter-class
One-Shot Open-World Group-based Re-id
![Page 129: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/129.jpg)
129
Local Relative Distance Comparison
constraining all the relative distance comparisonsaround the neighbourhood of a difference dataset
or
One-Shot Open-World Group-based Re-id
![Page 130: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/130.jpg)
130
Local Modelling: Remained Comparison
A
E
CD
B
F
One-Shot Open-World Group-based Re-id
![Page 131: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/131.jpg)
131
Local Modelling: Remained Comparison
A
E
CD
B
F
One-Shot Open-World Group-based Re-id
![Page 132: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/132.jpg)
132
Local Modelling: Removed Comparison
A
E
CD
B
F
One-Shot Open-World Group-based Re-id
![Page 133: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/133.jpg)
133
Local Modelling
ETHZ
CAVIAR
One-Shot Open-World Group-based Re-id
![Page 134: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/134.jpg)
134
A stochastic gradient algorithm
Compute Active SetThe local neighbourhood sets are updated at each step
One-Shot Open-World Group-based Re-id
![Page 135: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/135.jpg)
135
Individual Verification
One-Shot Open-World Group-based Re-id
![Page 136: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/136.jpg)
136
Individual Verification
One-Shot Open-World Group-based Re-id
![Page 137: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/137.jpg)
137
Individual Verification
One-Shot Open-World Group-based Re-id
![Page 138: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/138.jpg)
138
Individual Verification
One-Shot Open-World Group-based Re-id
![Page 139: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/139.jpg)
139
Individual Verification
One-Shot Open-World Group-based Re-id
![Page 140: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/140.jpg)
When Clothing Change? Bad Lighting?
![Page 141: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/141.jpg)
141
Depth-based Re-identification
Depth RE-ID
Ancong Wu, Wei‐Shi Zheng*, and Jian‐Huang Lai. Depth‐based Person Re‐identification. Asian Conference on Pattern Recognition, 2015, oral.
Ancong Wu, Wei‐Shi Zheng*, and Jian‐Huang Lai. Robust Depth‐based Person Re‐identification. Submitted to IEEE Transactions on Image Processing, 2016. (Minor Revision)
In these cases, appearance cues are not reliable.
Illumination change
Clothes change
![Page 142: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/142.jpg)
142
Which Body Part is ImportantThe Integrated Matching Scheme (IMS):(ISBA 2015)“Towards More Reliable Matching for Person Re-identification”
The upper-body is superior to other body parts
![Page 143: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/143.jpg)
143
Depth-based Re-identification
Depth RE-ID
• Within-patch Covariance
• Between-patch Covariance
• Eigen-depth feature
Eigen-depth feature is rotation invariant.
![Page 144: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/144.jpg)
144
Depth RE-ID Depth-based Re-identification
Extracting Eigen-depth feature converts covariance matrices on
Riemannian manifold to feature vectors in Euclidean space.
Theorem
ixjx
O
![Page 145: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/145.jpg)
145
Depth RE-ID
![Page 146: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/146.jpg)
146
Depth (Identification/ Re-identification)
Attention in the Dark: A Recurrent Attention Model for Person Identification, by Albert Haque Alexandre Alahi Li Fei-Fei @ CVPR 2016
Depth-based Person Re-identification, by AncongWu , Wei-Shi Zheng∗ , and Jian-Huang Lai, ACPR 2015. Journal VersionSubmitted
![Page 147: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/147.jpg)
147
Brief Introduction of ML for Biometrics ML for Person Re-identification
Distance Metric Learning View Change Invariant Features Partial Re-id Low Resolution Video-based Re-id Cross Scenario Transfer Open-world Modelling Depth Re-identification
Summary
Outline
![Page 148: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/148.jpg)
148
Summary
RE-ID Specific Distance Metric Learning
OPEN-WOLRDRE-ID(TPAMI 2016/CVPR2012)
Partial RE-ID(ICCV2015)
DEEP RE-ID(WACV2016
TPAMI Minor)
Video-basedRREID(CVPR2016/PR2011)
Relative Distance Comparison(TPAMI 2013, CVPR2011)
Cross-scenarioRE-ID(TCSVT 2016)
Depth RE-ID(ACPR2015, TIP Minor)
Multi-scaleMethod(ICCV2015)
![Page 149: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/149.jpg)
149
Summary Some thoughts of RE-ID
Not just a topic about image-image recognition Not just about a conventional classification problem Not just about a conventional retrieval problem Not just a machine learning task Interaction with operators: Human in the loop Long-term ……… Unsupervised Learning How to select the person you want to track?
o Activity/action of our works: ICCV 2013, CVPR 2015, ECCV2016, TIP 2015 (small group, early prediction, RGB-D)
![Page 150: Machine Learning for Person Identification](https://reader036.vdocument.in/reader036/viewer/2022090905/613c8ab5a9aa48668d4a30aa/html5/thumbnails/150.jpg)
150
VISITING MY HOME PAGE
http://isee.sysu.edu.cn/~zhwshi
http://isee.sysu.edu.cn
EMAIL ME:[email protected]
MORE INFO.