![Page 1: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/1.jpg)
Multiple View Based Multiple View Based 3D Object Classification 3D Object Classification
Using Ensemble Learning of Using Ensemble Learning of Local Subspaces (Local Subspaces (ThBT4.3ThBT4.3))
Jianing Wu, Kazuhiro Fukui
[email protected],[email protected]
Graduate school of Systems and Information Engineering
University of Tsukuba (Japan)
![Page 2: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/2.jpg)
2
AbstractAbstract
We proposed a statistical method for object classification based on multi-view.
The proposed method is an extension of MSM.
Problems of previous works (MSM, KMSM) has been solved.
We evaluated the classification performance of the proposed method and previous works.
![Page 3: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/3.jpg)
3
Table of ContentsTable of Contents
Backgrounds Multi-view classification and problems of existing
methods for nonlinear distribution
The proposed method Approximation by local subspaces and ensemble
learning
Experimental results Performance comparison using multi-view images of
objects
Summary
![Page 4: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/4.jpg)
4
Multi-view Based Object ClassificationMulti-view Based Object Classification
Multiple view is more beneficial for object classification than single.
View based approach does not need 3D model for classification.
Subspace related methods has been proposed for multi-view frame work.
![Page 5: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/5.jpg)
5
Existing view based approachesExisting view based approaches
Mutual Subspace Method (MSM)O.Yamaguchi, K.Fukui, K.Maeda: Face recognition using temporal image sequence. Proc.IEEE 3rd International Conference on Automatic Face and Gesture Recognition, pp.318-323,1998.
ProLow computation cost
ConCannot handle nonlinear distribution
![Page 6: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/6.jpg)
6
Multi-view Based Object ClassificationMulti-view Based Object Classification
Feature vectors from multi-view input are likely to have nonlinear distribution.
Subspace approximation therefore is not accurate.
![Page 7: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/7.jpg)
7
Existing view based approachesExisting view based approaches
Kernel Mutual Subspace Method (KMSM)H.Sakano, N.Mukawa: Kernel mutual subspace method for robust facial image recognition. Proc. 4th International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies, Vol.1, pp.245-248, 2000.
Project feature vectors to a high dimensional space to reduce their nonlinearity.
Pros: Can handle nonlinear distribution.
Cons: Consume more computation time, and this increases
at order N^2 as learning data increases. Some critical parameters need to be optimized
![Page 8: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/8.jpg)
8
Motivation of the Proposed MethodMotivation of the Proposed Method
Approximate nonlinear distribution. Divide the original distribution, and
approximate each subset.
Achieve comparable classification performance with KMSM, using less computation. Perform classification without kernel framework.
![Page 9: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/9.jpg)
9
Table of ContentsTable of Contents
Backgrounds Multi-view classification and problems of existing
methods for nonlinear distribution
The proposed method Approximation by local subspaces and ensemble
learning
Experimental results Performance comparison using multi-view images of
objects
Summary
![Page 10: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/10.jpg)
10
The Proposed MethodThe Proposed Method Divide the nonlinear distribution into several
subsets based on Euclidean distance. Nonlinearity of each subset is weaker.
We approximate each subset with local subspace.
![Page 11: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/11.jpg)
11
Ensemble ClassificationEnsemble Classification
Division number and dimension of each local subspace are parameters affect classification performance.
We construct local subspaces under multiple combination.
We assume each case as weak classifier and apply ensemble learning.
![Page 12: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/12.jpg)
12
The Proposed Method with WeightThe Proposed Method with Weight Each local subspace carries a weight coefficient based
on its classification performance. This coefficient weights the canonical angle. The coefficients are simply the accuracy rate of each
local subspace in preliminary experiments.
![Page 13: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/13.jpg)
13
The Proposed MethodThe Proposed Method
![Page 14: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/14.jpg)
14
Table of ContentsTable of Contents
Backgrounds Multi-view classification and problems of existing
methods for nonlinear distribution
The proposed method Approximation by local subspaces and ensemble
learning
Experimental results Performance comparison using multi-view images of
objects
Summary
![Page 15: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/15.jpg)
15
Classification ExperimentClassification Experiment
Compare the classification performance and computation cost of MSM, KMSM and the proposed method.
Use multi-view image data set ‘The ETH-80 Image Set’B.Leibe, B.Schiele: Analyzing appearance and contour based methods for object categorization. Proc. CVPR'03, Vol.2, pp.409-415, 2003
![Page 16: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/16.jpg)
16
Classification ExperimentClassification Experiment The dataset contains 8 classes, 10 objects
for each class. 41 points of
view for each object
Feature vector is the resized image (16x16)
![Page 17: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/17.jpg)
17
Classification ExperimentClassification Experiment
Use 164 images to learn for each class. (generate dictionary)
Evaluation input is images from an unknown object. (10 images/points of view)
By exchanging learning data and evaluation data (leave-one-out), we repeated experiment for 1640 times.
![Page 18: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/18.jpg)
18
Classification ExperimentClassification Experiment
Classification performance of each method
Method Accuracy Rate(%)
Separability
EER(%)
MSM 69.5 0.34 20
KMSM 87.2 0.41 15
Proposed method
86.5 0.44 14
Proposed method with weight
94.7 0.55 9
![Page 19: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/19.jpg)
19
Classification ExperimentClassification Experiment
The proposed method improved classification performance from MSM
The proposed method achieved comparable performance with KMSM
Classification performance of the proposed method was improved by introducing weight to each local subspace
![Page 20: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/20.jpg)
20
Classification ExperimentClassification Experiment
Computation cost of each method
Method Calculation Time (seconds per input)
Order
MSM 0.1 O(n)
Proposed method
0.4 O(n)
Proposed method with weight
0.4 O(n)
KMSM 3.1 O(n^2)
![Page 21: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/21.jpg)
21
Classification ExperimentClassification Experiment
The proposed method consumes less computation time compared with KMSM
The computation time of the proposed method increases in order N as learning data increases
![Page 22: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/22.jpg)
22
Classification ExperimentClassification Experiment Classification performance of weak classifiers
Proposed method
86.5 0.44 14
![Page 23: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/23.jpg)
23
Table of ContentsTable of Contents
Backgrounds Multi-view classification and problems of existing
methods for nonlinear distribution
The proposed method Approximation by local subspaces and ensemble
learning
Experimental results Performance comparison using multi-view images of
objects
Summary
![Page 24: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/24.jpg)
24
SummarySummary
We proposed a method to achieve comparable performance with KMSM by less calculation.
Classification performance is further improved by introducing weight to each local subspace
The advantages of the proposed method is shown with classification experiment of objects.
![Page 25: Multiple View Based 3D Object Classification Using Ensemble Learning of Local Subspaces ( ThBT4.3 ) Jianing Wu, Kazuhiro Fukui lacarte@cvlab.cs.tsukuba.ac.jplacarte@cvlab.cs.tsukuba.ac.jp,](https://reader035.vdocument.in/reader035/viewer/2022062421/56649d0c5503460f949df8a2/html5/thumbnails/25.jpg)
Thank YouThank You
Multiple View Based 3D Object Classification Using Ensemble Learning of Local
Subspaces (ThBT4.3)
Jianing Wu, Kazuhiro Fukui