Slide 1

PixelLaser: Range scans from image segmentation Nicole Lesperance 11 Michael Leece 11 Steve Matsumoto 12 Max Korbel 13 Kenny Lei 15 Zach Dodds 62 REU

Slide 2

Inspiration Horswill (polly) 94 Saxena (rccar) 05

Slide 3

Scans ? C. Plagemann et al., ICRA 2008 platform"omnicam" imageserrors...

Slide 4

Cheap! Expensive! Less information: 180 rays of distances More information: 640 columns of pixels, each of which we can get a range from But its harder (for a computer) to extract data from an image than from a laser range scan. Which is where we come in! Motivation

Slide 5

ClassifySegmentScans Maps! (in pictorial form!) obstacle ground (and localization) Overall Strategy

Slide 6

Training and Classifying hand-segmented imagefeatures, in kd-trees 194 191 211 3.2 25.6 4.1... 138 87 53 -1.14 8.6 1.4... filters

Slide 7

Nearest-neighbor classification RGB alone doesnt work well

Slide 8

Nearest-neighbors RGB + texture produces better segmentations

Slide 9

Classifier Variations Many attempts were made, but the overall accuracy for reasonable versions hovered around 95% 3-tree Classifier: used above, below, and line categories Genetic Algorithm: attempted to find the optimal weight values, but optimized for the set of images used to evolve the classifier. The weight-set did not generalize to other environments Different Environments: the classifier was tested in environments varying in difficulty

Slide 10

Overall Accuracy Results Regardless of the environment, the Classifier performs consistently

Slide 11

Hallway: Overall Accuracy 95.40 % Library Downstairs: Overall Accuracy 94.01 %

Slide 12

FLANN (Fast Library for Approximate Nearest Neighbors) C++ knn implementation with variable precision levels Decreased kd-tree lookup time by 2 orders of magnitude Allows for more complex real-time segmentation algorithms Allows the robot to autonomously wander in real time using only the segmenter Marius Muja: Our hero

Slide 13

Now Arriving atSegmentation! Uses the information from the classifier to draw the line between the ground and obstacles Sends the information to the scanner to make laser range scans Classify Segment Scan Map

Slide 14

Initial approaches (1)bottom-up (2)patch pairs (3)multiresolution

Slide 15

Transition Segmentation Uses transition strength rather than classification Assigns transition strength based on the certainty that the below patch is traversable and the top patch is not Takes the difference of the above strength of the top patch and the below strength of the bottom one Issues Misclassifications can result in incorrectly strong transitions

Slide 16

Transitions using a Line Tree The strongest transitions are found and the area closest matching the line category is chosen

Slide 17

Seam Carving Start at one end of the image and find the strongest transition Assign a cost to deviating from the line height in the previous column

Slide 18

Smoothing Certainty Locally Calculate certainties for all patches in the image Apply averaging filter on the certainties to suppress rogue misclassifications Look for the strongest transition in each column and draw line

Slide 19

Edge Detection Global image edge detection is not useful, but snapping to the nearest edge in the strongest transition area reduces segmentation error

Slide 20

Results Sprague LabSprague First FloorLibra Hallways Multi-resolution9.6716.2419.75 Smooth7.445.1022.11 Transition Line4.124.5214.27 Sprague LabSprague First FloorLibra Hallways Multi-resolution12.2816.0564.52 Smooth11.089.8130.15 Transition Line11.018.7222.78 Average Pixel Error per Column Average Distance Error per Column (inches)

Slide 21

Results - Moving Average Pixel Error per Column Sprague LabSprague First FloorLibra Hallways Multi-resolution8.8611.1314.10 Smooth3.627.9910.34 Transition Line2.796.0814.01 Sprague LabSprague First FloorLibra Hallways Multi-resolution27.3435.1055.34 Smooth17.6433.0159.18 Transition Line19.6836.4575.28 Average Distance Error per Column (inches)

Slide 22

From segmentation to scan pixel row number range row-to-range map scan segmentation

Slide 23

Scan Matching Match two consecutive laser scans and find the corresponding transformation Can be used to correct poor odometry

Slide 24

SLAM Map Building Combine all laser scans to buildmaps using CoreSLAM (simultaneous localization and mapping)

Slide 25

Monte Carlo Localization Create random list of possible locations of robot Monte Carlo Localization Cycle: Move: Move all points based on change in odometry Sense: Compare the laser scan from segmented image with laser scans at possible locations Resample: Redistribute points based on probability at each location

Slide 26

Monte Carlo Localization

Slide 27

Point and Click Navigation Given a known map, we want to be able to click on a location in the map and have the robot navigate there The current implementation works only if there is a straight, uninterrupted path between the current location and the destination (havent implemented a path planning algorithm) Things to think about include navigation around obstacles and odometry correction

Slide 28

Autonomous Wandering

Slide 29

Future Work Segmenting with a projected laser level line possibly to be used for automated training as well Stabilizing the camera keeping the horizon constant while moving is almost impossible Speeding up the wandering speed a more stable camera would help with this Autonomous mapping

Slide 30

Questions?

Slide 31

Segmentation classification + confidencesegmentations

Slide 32

../TrainingImages/Playspacepswo13Patches/00029/randomBelow/0009.png 194.2575 191.4525 211.4775 195.0 192.0 212.0 8.67070895314 8.76885076564 7.29105573631 211.4775 191.4525 194.2575 212.0 192.0 195.0 7.29105573631 8.76885076564 8.67070895314 3113.18473022 2918.61969259 194.707808243 -8.2951355554e-09 -2.99999999989 3.48871383089 -0.1821058002 1.05861940504 0.798119246226 0.142200402323 1.37643522494 1.51710154734 - 3.21108530262 25.6053670274 4.08979546594 0.20963020652 3.42407524192 7.98287776033 0.655198704147 1.40172476457 1.2862398764 -0.068793854887 7.97682628048 3.32687477389 -0.83219461218 2.58844742173 1.83432937157 -0.239667915222 1.22096756236 3.56803311807 0.0253655120168 0.415881369811 7.11907142389 - 0.829352050768 1.14166676605 4.1432245279 -0.33452013644 -0.0121915477333 6.30267114206 -0.714448599523 0.711384510758 3.38445323448 -0.566103623094 -0.00278409428233 5.92992221434 -0.932328715083 1.17018711051../TrainingImages/Playspacepswo13Patches/00026/randomabove/0007.png 138.1075 87.9525 53.4725 138.0 88.0 53.0 3.65184114523 3.44532200962 5.05512054752 53.4725 87.9525 138.1075 53.0 88.0 138.0 5.05512054752 3.44532200962 3.65184114523 1584.29001984 1485.27385943 99.0473083038 2.67691915514e-09 -2.99999999998 0.839568062542 -0.0913685033632 5.68745353063 0.334375851147 -0.0494906892805 5.40416976808 0.617058294109 -1.14642893693 8.69695694778 1.38147297404 -0.87936321568 4.10330225187 1.59491339286 -0.69926257399 0.679234808869 0.526708837532 -0.253691320988 7.92226252642 0.619482921372 -0.155302887182 1.51598656224 0.455410613623 0.896134364593 4.90251778762 1.44298148203 -1.32247289798 1.54652876227 1.82381743704 1.10446810418 0.850926709651 1.23981013329 -0.346508022218 -0.252958061091 1.53636914298 1.26906228604 1.09496630534 1.45689240135 1.4011501228 1.85701734954 1.68840166501 1.0053643485 0.471348780351 194 191 211 3.2 25.6 4.1 138 87 53 -1.14 8.6 1.4

Slide 33