landmark selection for vision-based navigation

Landmark SelectionLandmark Selectionfor Vision-Based Navigationfor Vision-Based Navigation

Pablo L. SalaPablo L. SalaJoint work with Robert Sim, Ali Shokoufandeh and Sven DickinsonJoint work with Robert Sim, Ali Shokoufandeh and Sven Dickinson

To be presented in IROS 2004To be presented in IROS 2004

September 17September 17thth, 2004, 2004

Robot NavigationRobot Navigation

[Leonard and Durrant-Whyte][Leonard and Durrant-Whyte]

•Where am I?Where am I?

•Where am I going?Where am I going?

•How do I get there?How do I get there?



•Where am I?Where am I? Localization Localization

•Where am I going?Where am I going?





•Where am I going?Where am I going? Goal Identification Goal Identification





•Where am I going?Where am I going? Goal Identification Goal Identification

•How do I get there?How do I get there? Path-planning Path-planning

Landmark-Based NavigationLandmark-Based Navigation

• What makes a good landmark?What makes a good landmark?



• Distinctiveness (does it tell me where I am?)Distinctiveness (does it tell me where I am?)




• Wide VisibilityWide Visibility





• How do we select good landmarks?How do we select good landmarks?






• ManuallyManually







• AutomaticallyAutomatically







• Automatically… but how?Automatically… but how?








• Store every landmark visible at each location Store every landmark visible at each location (costly!)(costly!)








• Store every landmark visible at each location Store every landmark visible at each location (costly!)(costly!)

• Find smallest subset of landmarks that supports Find smallest subset of landmarks that supports reliable navigation (optimal!) reliable navigation (optimal!)

View-Based Robot NavigationView-Based Robot Navigation

Off-lineOff-lineExplorationExploration

LandmarkLandmarkDatabaseDatabase

ConstructionConstruction

On-lineOn-lineLocalizationLocalization


• Collection of images acquired at known discrete points in Collection of images acquired at known discrete points in pose space. Pose recorded and image features extracted pose space. Pose recorded and image features extracted and stored in database.and stored in database.










Four Four features are features are needed in needed in this set.this set.






Four Four features are features are needed in needed in this set.this set.

Only two features Only two features needed.needed.

Our goal is to find this Our goal is to find this decomposition.decomposition.


• Current pose is estimated using the locations of a small Current pose is estimated using the locations of a small number of features in the current image, matched against number of features in the current image, matched against their locations in two model views.their locations in two model views.





Intuitive Problem FormulationIntuitive Problem Formulation

OutlineOutline

• Problem FormulationProblem Formulation

• ComplexityComplexity

• Heuristic MethodsHeuristic Methods

• Results on Synthetic and Real Results on Synthetic and Real ImagesImages

• ConclusionsConclusions

A Graph Theoretic FormulationA Graph Theoretic Formulation

Problem DefinitionProblem Definition::

The The -Minimum Overlapping Region Decomposition -Minimum Overlapping Region Decomposition ProblemProblem ( (-MOVRDP) for a world instance <-MOVRDP) for a world instance <GG=(=(VV,,EE), ), FF, {, {vv}}

vvVV> consists of finding a minimum size > consists of finding a minimum size -overlapping -overlapping

decomposition decomposition DD = { = {RR11, …, , …, RRdd} of } of VV into into

regions such that:regions such that:







Theorem 1:Theorem 1: A A -MOVRDP can be reduced to an equivalent 0--MOVRDP can be reduced to an equivalent 0-MOVRDP, and the solution to this latter problem can be MOVRDP, and the solution to this latter problem can be extended to a solution for the original problem.extended to a solution for the original problem.







Theorem 1:Theorem 1: A A -MOVRDP can be reduced to an equivalent 0--MOVRDP can be reduced to an equivalent 0-MOVRDP, and the solution to this latter problem can be MOVRDP, and the solution to this latter problem can be extended to a solution for the original problem.extended to a solution for the original problem.

Theorem 2:Theorem 2: The decision problem <0-MOVRDP, The decision problem <0-MOVRDP, dd> is NP-> is NP-complete.complete.

(Proof by reduction from the Minimum Set Cover Problem.)(Proof by reduction from the Minimum Set Cover Problem.)

Heuristic Methods for 0-MOVRDPHeuristic Methods for 0-MOVRDP• 0-MOVRDP is intractable.0-MOVRDP is intractable.

• Optimal Optimal decomposition not needed in practice.decomposition not needed in practice.

• We developed and tested six greedy approximation algorithms.We developed and tested six greedy approximation algorithms.




Algorithm A.xAlgorithm A.x: : OO(|(|VV||22||FF|)|)

Features commonly visible in region:Features commonly visible in region:

kk = 4 = 4





Features commonly visible in region: Features commonly visible in region: 2525

kk = 4 = 4






kk = 4 = 4

Heuristic Methods for 0-MOVRDPHeuristic Methods for 0-MOVRDPAlgorithmsAlgorithms

B.xB.x and and CC::

OO((kk||VV||22||FF|)|)

kk = 5 = 5

Features commonly visible in region: Features commonly visible in region:


B.xB.x and and CC::

OO((kk||VV||22||FF|)|)

kk = 5 = 5


ResultsResultsSimulated DataSimulated Data

ResultsResultsSimulated Data (cont.)Simulated Data (cont.)


World SettingsWorld Settings

• Two types of Worlds: Irregular (Two types of Worlds: Irregular (IrregIrreg) and Rectangular ) and Rectangular ((RectRect).). average diameter: 40m.average diameter: 40m. pose space sampled at 50 cm intervals.pose space sampled at 50 cm intervals. average number of sides: 6.average number of sides: 6. average number of obstacles: 7.average number of obstacles: 7.


World SettingsWorld Settings

• Two types of Worlds: Irregular (Two types of Worlds: Irregular (IrregIrreg) and Rectangular ) and Rectangular ((RectRect).). average diameter: 40m.average diameter: 40m. pose space sampled at 50 cm intervals.pose space sampled at 50 cm intervals. average number of sides: 6.average number of sides: 6. average number of obstacles: 7.average number of obstacles: 7.

• Two types of Features: Two types of Features: Short-RangeShort-Range and and Long-RangeLong-Range.. visibility range visibility range NN(0.65, 0.2) to (0.65, 0.2) to NN(12.5, 1) m,(12.5, 1) m,

and angular range and angular range NN(25, 3) degrees.(25, 3) degrees. Visibility range Visibility range NN(0.65, 0.2) to (0.65, 0.2) to NN(17.5, 2) m,(17.5, 2) m,

and angular range and angular range NN(45, 4) degrees.(45, 4) degrees.

Results (cont.)Results (cont.)Simulated Data (cont.)Simulated Data (cont.)

• 300 randomly generated worlds300 randomly generated worlds

• Runtime of few secondsRuntime of few seconds

• Avg. size of regions depends on stability of features in pose space.Avg. size of regions depends on stability of features in pose space.

• Number of regions increases as avg. size of regions decreases.Number of regions increases as avg. size of regions decreases.

• Alg. B.2 achieved the best results.Alg. B.2 achieved the best results.

SettingSetting WorldWorld FeatureFeature

11 RectRect Short-Short-RangeRange

22 RectRect Long-Long-RangeRange

33 IrregIrreg Short-Short-RangeRange

44 IrregIrreg Long-Long-RangeRange

AlgorithmsAlgorithms A.1A.1

A.2A.2

A.3A.3

B.1B.1

B.2B.2

CC

Results (cont.)Results (cont.)Real DataReal Data

We applied the best-performing algorithm (B.2) to real We applied the best-performing algorithm (B.2) to real feature visibility data.feature visibility data.

00 9090

180180 270270

Results (cont.)Results (cont.)Real DataReal Data

Experiment 1Experiment 1

• Data collected in 2m Data collected in 2m 2m area. 2m area.

• Sampled at 20 cm intervals.Sampled at 20 cm intervals.

• Total of 46 visible features.Total of 46 visible features.

• Camera at a fixed orientation (looking forward).Camera at a fixed orientation (looking forward).

• Features were extracted using the Kanade-Lucas-Tomasi Features were extracted using the Kanade-Lucas-Tomasi operator.operator.

• Parameters used: Parameters used: = 0, = 0, kk = 4. = 4.

Results (cont.)Results (cont.)


• Data collected in 6m Data collected in 6m 3m area.3m area.



• Camera at 0, 90, 180, and 270Camera at 0, 90, 180, and 270

degree orientations.degree orientations.

• Lowe’s SIFT features.Lowe’s SIFT features.

Results (cont.)Results (cont.)


• Data collected in 6m Data collected in 6m 3m area.3m area.



• Camera at 0, 90, 180, and 270Camera at 0, 90, 180, and 270

degree orientations.degree orientations.

• Lowe’s SIFT features.Lowe’s SIFT features.

Typical Feature Visibility Typical Feature Visibility RegionsRegions

Results (cont.)Results (cont.)Experiment 2 (cont.)Experiment 2 (cont.)

• k=4, k=4, =0=0


• k=4, k=4, =1=1


• k=10, k=10, =0=0


• k=10, k=10, =1=1

ConclusionsConclusions

• We have introduced a novel graph theoretic We have introduced a novel graph theoretic formulation of the landmark acquisition problem, formulation of the landmark acquisition problem, and have established its intractability.and have established its intractability.

• We have explored a number of greedy We have explored a number of greedy approximation algorithms, systematically testing approximation algorithms, systematically testing them on synthetic worlds and demonstrating them them on synthetic worlds and demonstrating them on two real worlds. on two real worlds.

• The resulting decompositions find large regions in The resulting decompositions find large regions in the world in which a small number of features can the world in which a small number of features can be tracked to support efficient on-line localization.be tracked to support efficient on-line localization.

• The formulation and solution are general, and can The formulation and solution are general, and can accommodate other classes of image features.accommodate other classes of image features.

Future WorkFuture Work

• Use feature tracking during the image collection Use feature tracking during the image collection stage to achieve larger areas of visibility for each stage to achieve larger areas of visibility for each feature. (Maintain equivalence classes of features feature. (Maintain equivalence classes of features in the DB.)in the DB.)

• Integrate the image collection phase with the Integrate the image collection phase with the region decomposition stage to yield an on-line region decomposition stage to yield an on-line process for simultaneous exploration and process for simultaneous exploration and localization (SLAMB).localization (SLAMB).

• Path planning through decomposition space, Path planning through decomposition space, minimizing the number of region transitions in a minimizing the number of region transitions in a path.path.

• Detect and cope with environmental change.Detect and cope with environmental change.

• Compute the performance guarantee of our Compute the performance guarantee of our heuristic methods and provide tight upper bounds heuristic methods and provide tight upper bounds on the quality of our solution compared to the on the quality of our solution compared to the optimal.optimal.

landmark selection for vision-based navigation

Documents

good landmarks

landmark visible

wide visibilityhow

pose space

discrete points

image features

lin of images

smallest subset of landmarks