dynamic 3d scene analysis for acquiring articulated scene models...

1

Dynamic 3D Scene Analysis for Acquiring Articulated Scene Models

Agnes Swadzba, Niklas Beuter, Sven Wachsmuth, Franz Kummert

Applied Informatics, Bielefeld University, Germany

ICRA 2010

2

Overview


•  Motivation

•  System overview

•  Experiments & Results

3

Human-robot interaction

•  Spatial awareness essential for (mobile) robotics

•  Segment the scenery in meaningful parts

•  Static scene -> Room classification

•  Movable objects -> Functional connection

•  Moving objects -> Interaction partners

•  Subsequent processing suffers from deep knowledge about the scenery

Motivation


4

Motivation – Articulated scene model


Static Scene

Movable Objects

Moving Objects

Observing all parts as an articulated scene model enhances the detection of each part in exchange

-Walls -Cupboards - …

-Persons -Robots - …

-Chairs -Doors - …

5

•  Swissranger Time-of-flight camera to receive the scene in 3D

•  2D amplitude image (gray image)

•  6D data consisting of 3D points and 3D velocity

System requirements


6

•  Distance adaptive median filtering of 3D points

•  3D dense velocity computation and filtering

Preprocessing


7

•  Observe differences in subsequent frames

•  Identify moving and movable objects through knowledge about the static scene

•  Update the static scene by physical principle

System overview


!"#$%&'$ !"#$%&'$()*&#+,-)#.,'$

/,$,'$0)1-2-$%&'3014

&/&5$

6%&(,-7!

!$&$0'-!',1,-8!

!$&$0'-!',1,-8!"#

!$&$0'--5&%$!

()*&#+,-)#.,'$-5)01$!-9!

5)$,1$0&+-/:1&(0'-5&%$!

)#.,'$!-$)-%,()*,

;$

<$

&1&+:!,1,=-!$&$0'-

!',1,-5&%$!

+0!$-)6-)#.,'$-5)01$!4%)"5,/-#:-$0(,-!

>>>

8

Detection

•  Simplify scene by subtraction of previously known static parts St-1

•  Cluster similar velocities and location of remaining potential points Pt

•  Using a cylindric object model, moving objects can be found

Tracking

•  Track objects through an hybrid particle filter with meanshift

Detection and Tracking of moving objects


9

Detection and Tracking of moving objects


Potential dynamic points Pt

Probability density function Found objects

10

•  Observe differences in subsequent frames

•  Identify moving and movable objects through knowledge about the static scene

•  Update the static scene by physical principle

System overview


!"#$%&'$ !"#$%&'$()*&#+,-)#.,'$

/,$,'$0)1-2-$%&'3014

&/&5$

6%&(,-7!

!$&$0'-!',1,-8!

!$&$0'-!',1,-8!"#

!$&$0'--5&%$!

()*&#+,-)#.,'$-5)01$!-9!

5)$,1$0&+-/:1&(0'-5&%$!

)#.,'$!-$)-%,()*,

;$

<$

&1&+:!,1,=-!$&$0'-

!',1,-5&%$!

+0!$-)6-)#.,'$-5)01$!4%)"5,/-#:-$0(,-!

>>>

11

3D scene analysis


!"#$%&'$ !"#$%&'$()*&#+,-)#.,'$

/,$,'$0)1-2-$%&'3014

&/&5$

6%&(,-7!

!$&$0'-!',1,-8!

!$&$0'-!',1,-8!"#

!$&$0'--5&%$!

()*&#+,-)#.,'$-5)01$!-9!

5)$,1$0&+-/:1&(0'-5&%$!

)#.,'$!-$)-%,()*,

;$

<$

&1&+:!,1,=-!$&$0'-

!',1,-5&%$!

+0!$-)6-)#.,'$-5)01$!4%)"5,/-#:-$0(,-!

>>>

Results: •  Moving objects and their trajectory •  Articulated scene parts like chairs or doors •  Static scene parts

12

•  The setup is tested on 14 sequences:

•  4 Scenarios of a person moving objects:

•  Evaluate found static scene parts to ground trouth static scene (Mgt) in pixel error and standard deviation

•  Evaluation with 4 algorithms:

•  Madapt, Mmean, Mmpix, Mtrack

Experiments and results


•  Rearranging teddy bears

•  Searching a teddy bear

•  Tidy up

•  Opening and closing doors

13



S1R1 S1R2 S1R3 S1R4 S1R5 S1R6 S2R1 Mmean 103±177 106±204 124±222 157±284 142±278 147±262 95±187 Mpix 64±121 74±184 79±185 111±216 99±230 95±193 71±155 Mtrack 71±166 108±209 78±189 97±212 79±308 98±219 84±182 Madapt 18±59 19±47 21±61 24±78 24±68 21±55 20±96

S2R2 S3R1 S3R2 S4R1 S4R2 S4R3 S4R4 Mmean 108±147 89±105 85±183 219±403 321±639 234±451 246±594 Mpix 80±118 63±145 61±125 163±328 299±635 229±588 229±588 Mtrack 85±140 71±141 134±712 51±165 74±218 356±677 246±601 Madapt 16±37 20±58 22±52 14±26 75±319 18±64 98±404

14



15



16

Work in Progress:

•  Enhancement of moving object detection via image features

•  Combination of the saccadic views via ICP

•  Scene observation during movement via motion compensation

•  Labeling of articulated parts via dialog

Short Outlook


17

... for your attention

•  Questions?

Thank you ...


dynamic 3d scene analysis for acquiring articulated scene models...

Documents