an integrated feature selection strategy for monocular...
TRANSCRIPT
![Page 1: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/1.jpg)
An Integrated Feature Selection
Strategy For Monocular SLAM
Presenter: Kuan-Ting Yu
Advisor: Li-Chen Fu
NTU Intelligent Robot Lab
Industrial Technology Research Institute
September, 2011
1
![Page 2: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/2.jpg)
Outline
Introduction
Feature Extraction
Bottom-up Feature Selection
Top-down Feature Selection
Experimental Results
Conclusions
2
![Page 3: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/3.jpg)
Outline
Introduction
Feature Extraction
Bottom-up Feature Selection
Top-down Feature Selection
Experimental Results
Conclusions
3
![Page 4: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/4.jpg)
Introduction
To perform desired tasks in indoor environments,
how to estimate robot’s position and map about
surroundings is a critical problem.
4
![Page 5: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/5.jpg)
SLAM
Simultaneous localization and mapping (SLAM)
Incrementally build a map of this environment while
simultaneously determining its location
Given:
1. Robot’s odometry
2. Observations of nearby features
Estimate:
1. Location of the robot
2. Map of features
5
![Page 6: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/6.jpg)
Sensors
6
![Page 7: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/7.jpg)
vSLAM
Generic SLAM
Landmark extraction
Data association
State predict
State update &
landmark update
Vision-based SLAM
Interest points (features)
Interest points matching
7
Probabilistic framework
![Page 8: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/8.jpg)
vSLAM Challenge
Huge amount of features
Degrade performance
Cause mismatch
8
![Page 9: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/9.jpg)
Solution Idea
Feature selection strategy
Remove useless features
Keep good features
9
![Page 10: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/10.jpg)
System Overview
1. Landmark extraction
2. Data association
3. State predict
4. State update &
landmark update
10
![Page 11: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/11.jpg)
Outline
Introduction
Feature Extraction
Bottom-up Feature Selection
Top-down Feature Selection
Experimental Results
Conclusions
11
![Page 12: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/12.jpg)
What is Feature?
Features (interest points):
Easy to find their correspondences
Desired features:
Distinctive: outstanding, easily matched
Invariant: invariant to scale, viewpoint change
12
?
![Page 13: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/13.jpg)
Feature Detectors
Scale Invariant Feature Transform (SIFT) [Lowe, 2004] is
one of the best feature detectors,
Speeded Up Robust Features (SURF) [Bay, et al., 2006]
SURF has good performance as SIFT and faster
SIFT SURF
Images
Extraction time 863.998 ms 267.634 ms
13
but slow
![Page 14: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/14.jpg)
Feature Extraction Challenge
Hundreds of SURF points are extracted
Increase computational time
Need higher data storage
But, fewer features are desired in practice
Remove useless features
Keep repeatable features
A feature selection strategy is necessary
14
![Page 15: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/15.jpg)
Feature Selection
Human never process the whole image at once
Focus on some regions of interest (ROIs)
A natural solution — visual attention system
15
Bottom-up approach Top-down approach
Image-driven Knowledge-driven
![Page 16: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/16.jpg)
Outline
Introduction
Feature Extraction
Bottom-up Feature Selection
Top-down Feature Selection
Experimental Results
Conclusions
16
![Page 17: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/17.jpg)
Feature Selection – Bottom-up Approach
Visual attention system
Saliency model
[Itti et al., 1998]1
Replace the original RGB
color space with CIE L*a*b
Mimic color opponencies
of human vision
17
[1] L. Itti, C. Koch, and E. Niebur, “A Model of Saliency-Based Visual Attention for Rapid Scene Analysis,” PAMI, 1998.
![Page 18: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/18.jpg)
Feature Selection – Bottom-up Approach
Results of modified saliency model
18
1( ( ))3
Sal N C I O
![Page 19: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/19.jpg)
Outline
Introduction
Feature Extraction
Bottom-up Feature Selection
Top-down Feature Selection
Experimental Results
Conclusions
19
![Page 20: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/20.jpg)
Feature Selection - Top down approach
Sometimes, bottom-up ROIs are not enough
For example:
Integrates top-down approach to achieve flexible
feature selection
20
![Page 21: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/21.jpg)
Feature Selection - Top down approach
Human robot interaction (HRI) can be applied to
object learning
Communication with the robot
Pointing with intelligent devices
21
![Page 22: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/22.jpg)
Feature Selection - Top down approach
Robot redetects known objects process
Based on an object
recognition algorithm
RANSAC
(Reject inconsistent matches)
Compute Homography
22
'
'
1 1
i i
i i i
x x
s y H y
1 1,x y
1 1', 'x y
2 2,x y
2 2', 'x y
H
=
![Page 23: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/23.jpg)
Solving the homography matrix, then we project the
four end-points to determine top-down ROI
23
Feature Selection - Top down approach
![Page 24: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/24.jpg)
Feature Selection
Merge two ROIs to obtain versatile features
24
![Page 25: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/25.jpg)
Outline
Introduction
Feature Extraction
Bottom-up Feature Selection
Top-down Feature Selection
Experimental Results
Conclusions
25
![Page 26: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/26.jpg)
Experiment Setting
Pioneer 3DX
Logitech webcam V-UBH44
26
Room size: 6m*8m
![Page 27: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/27.jpg)
SLAM Map
27
![Page 28: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/28.jpg)
Localization Error Comparison
28
TDBU: the proposed integrated bottom-up and top-down selection TD: top-down BU: bottom-up NO: without selection
Variance
Average
![Page 29: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/29.jpg)
Time Comparison
29
TDBU: the proposed integrated bottom-up and top-down selection TD: top-down BU: bottom-up NO: without selection
![Page 30: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/30.jpg)
Matching Comparison 1/2
30
![Page 31: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/31.jpg)
Matching Comparison 2/2
31
![Page 32: An Integrated Feature Selection Strategy For Monocular ...people.csail.mit.edu/peterkty/pub/KuanTing_CVGIP2011...An Integrated Feature Selection Strategy For Monocular SLAM Presenter:](https://reader036.vdocument.in/reader036/viewer/2022081403/60a72305406fa54ba440032d/html5/thumbnails/32.jpg)
Conclusions
We propose an integrated feature selection strategy
based on visual attention system for bearing-only
SLAM with EKF
Reduce computation time to 62%
Reduce localization error to 89%
Combining bottom-up and top-down approach to
construct ROIs allow us to
Select salient and useful features
Improve data association
32