christian laugier – e-motion team-project 1 the e-motion team-project « geometry and probability...

Christian LAUGIER – e-Motion Team-Project 1

The The e-Motione-Motion Team-Project Team-Project

« Geometry and Probability for motion and action »« Geometry and Probability for motion and action »

Inria Rhône-Alpes & Gravir laboratory (UMR 5527)

Scientific leader : Christian LAUGIER


Team-Project membersTeam-Project members (year 2004)(year 2004)

• Permanent staffPermanent staff– Christian Laugier, DR2 Inria (Scientific leader)– Emmanuel Mazer, DR2 Cnrs (External collaborator currently in our start-up « Probayes »)– Pierre Bessiere, CR1 Cnrs– Thierry Fraichard, CR1 Inria– Sepanta Sekhavat, CR1 Inria (Currently in Iran for 2 years)– Olivier Aycard, MC UJF– Anne Spalanzani, MC UPMF

• Invited researchers, Postdocs, and EngineersInvited researchers, Postdocs, and Engineers– Juan-Manuel Ahuactzin– Olivier Malrait– Kamel Mekhnacha– Jorge Hermosillo– Christophe Coué

• PhD studentsPhD students– 7 theses defended in 2003: J. Diard, C. Mendoza, R. Garcia, J. Hermosillo, F. Large, C. Coué, K. Sundaraj– 8 PhD students: C. Pradalier, C. Koike, M. Amavicza, R. Lehy, F. Colas, D. Vasquez, P.Dangauthier, M. Yguel– 3 PhD students co-directed : M. Kais (rocquencourt), S. Petti (rocquencourt), B. Rebsamen (NUS)– 5 Master students : Christopher Tay, Alejandro Vargas, Ruth Lezama, Christophe Braillon, Julien Burlet


Objectives & motivationsObjectives & motivations

• Scientific challenge :Scientific challenge : To develop new models for constructing « artificial systems » having sensing, decisional, and acting capabilities sufficiently efficient and robust for making them really operational in open (i.e. large & weakly structured) and dynamic environments

• Practical objective :Practical objective : To built operational (i.e. scalable) systems having the capability to « share our living space » in some selected application domains (e.g. transportation, personal robots …).

• Current technological context :Current technological context : (1) Continuous & fast growing of computational power; (2) Fast development of micro & nano technologies (mechatronics); (3) Increasing impact of information & telecom technologies on our everyday life (ambiant intelligence)

• Motivations & difficulties :Motivations & difficulties : Instead of promises & impressive advances in robotics in the last decade, almost no advanced robots are currently evolving around us !

=> Not reliable enough, Weak reactivity and low efficiency (real time constraints), Exhibit “cybernetic” behaviors, Quite difficult to program (with no real learning capabilities)


A large spectrum of potential applicationsA large spectrum of potential applications

Virtual autonomous agents Virtual autonomous agents (natural interaction with humen)

Rehabilitation & Medical robotsRehabilitation & Medical robots

Future carsFuture cars(driving assistance & autonomous driving)

““Personal Robot Assistant”Personal Robot Assistant”

Main considered applications

=> Transport & logistics, Services, Rehabilitation & Medical care, Entertainment …


Cooperation & ContractsCooperation & Contracts

• International cooperationInternational cooperationJapan (Riken), Singapore (NTU, NUS, SIMTech), USA (UCLA, Stanford, MIT), Mexico (IESTM Monterrey, UDLA Puebla), Europe (EPFL, Univ college of

London)

• Industrial cooperationIndustrial cooperationRobosoft, Renault, PSA, XL-Studio, Aesculap-BBrown, Teamlog, KelkooStartups: ITMI, Getris Images, Aleph Technologies, Aleph Med, Probayes (oct. 2003)

• R&D contractsR&D contracts– Industrial : ACI « Protege », Priamm « Visteo », RNTL « Amibe », Kelkoo– National : Robea (EVbayes, Parknav), Predit (Arcos, MobiVip, Puvame)– Europe : NoE « Euron », IST-FET « Biba », IST « Cybercars », IST « Prevent»


SScientific approachcientific approach

• Our approachOur approach To focus on « complexity » and « incompleteness » problems (scalability & real world ) We guess that this can be achieved by combining geometrical and probabilistic approaches

• Main difficultiesMain difficulties– Previous approaches on AI & Robotics have shown their limitations

=> Logics (70’s), Geometry (80’s), Random search (90’s), purely Reactive Architectures (90’s)– The real world is too complex for being fully modeled using classical tools (inparticular: incompleteness & uncertainty)

=> Additional methods are required (e.g. probabilistic programming)=> Biologic inspiration could bring some help (sensori-motors systems, internal representationsfor motions… which seems mostly based on probabilistic laws)

• Required technological breakthroughsRequired technological breakthroughs– Motion & action autonomy in a complex dynamic world

=> Incremental world modeling, time-space dimension, prediction & estimation of obstacles motions– Increased robustness & safety of navigation systems (perception & control)

=> Dealing with incompleteness & uncertainty– “Easy” programming & system adaptativity

=> Self learning capabilities & behaviors coding and mixing


Two complementary reasoning processesTwo complementary reasoning processes

Mastering the complexity by using the right reasoning level & incremental approaches

Taking explicitly into account the hidden variables at the reasoning level

Dynamic worldDynamic world

Space & MotionSpace & MotionModelsModels

Analytical & Statistical dataSensing data

Geometric & Kinematic reconstruction,SLAM

Motion prediction

Motion planMotion plan& Navigation controls& Navigation controls

Constrained Motion PlanningDifferential Flatness,Velocity Space, ITP

IncompletnessIncompletness

UncertaintyUncertainty

Preliminary Knowledge+

Experimental Data=

Probabilistic Representation

Maximum EntropyPrinciple

ii PP log

Queries &Queries &Decision ProcessDecision Process

Bayesian Inference(NP-Hard [Cooper 90]Heuristics & Optimization)

P(AB|C)=P(A|C)P(B|AC)=P(B|C)P(A|BC)P(A|C)+P(¬A|C) = 1

Christian LAUGIER – e-Motion Team-Project

Bayesian Programming (principle)Bayesian Programming (principle)D

escr

ipti

onD

escr

ipti

onQ

ues

tion

Qu

esti

on

Bay

esia

n P

rogr

amB

ayes

ian

Pro

gram

• Specification Specification => Preliminary Knowledge – Set of relevant probabilistic variables

– Parametric forms assigned to some of the terms appearing in the decomposition (gaussian, uniform …)

– Decomposition of the joint distribution (=> efficient way to compute it)

nXXX ,...,, 21

kkn RLRLLXXX |P...|P|P|...P 22121

iR

ii LRL i ,

f|P

• Identification Identification (of some of the terms of the decomposition)

=> Experimental Data P Li | Ri P Li | Ri

=>Inference (i.e. answer to the question) carried out by an “inference engine” (symbolic simplification + numerical computation)

Unknown

nXXXKnownSearch |...P1

|P 21

e.g. using ProBT software (Probayes)


Research axesResearch axes• Multi-modal modeling of space & motionMulti-modal modeling of space & motion

– Incremental world modeling– Prediction & estimation of obstacles motions– Sensori-motors maps

Sensori-motors space(bayesian maps)Dynamic SLAM

q

q

t

0,, ii qq tqq gg ,,

• Motion planning in a dynamic worldMotion planning in a dynamic world– Iterative trajectory planning (ITP)– Instantaneous escaping trajectories (Velocity space)– States of unavoidable collisions (State-time space)

NH constraints

Space-Time constraints

Sensory data fusion(situation analysis)

Learning reactive behaviors

• Decision in an uncertain world (Bayesian inference)Decision in an uncertain world (Bayesian inference)– Bayesian programming tools– Automatic learning & Entropy maximization– Biological inspiration


Autonomous navigation & Easy robot programmingAutonomous navigation & Easy robot programming

SLAMSLAM++

NH Motion planningNH Motion planning++

Reactive nReactive navigationavigation

Several functionalities (learned and downloaded) have to be combinedIncremental world modeling & localization + Motion planning + Autonomous sensor-based navigation

[Pradalier & Hermosillo 03]


Bayesian reactive behaviorsBayesian reactive behaviors

Current work: More complex situations & Learning, Better integration with Control

=> Controlling the vehicle using a probability distribution on v e.g. reducing speed and/or modifying steering angle for avoiding a pedestrian or a car

Gaussian functionCommand fusionCommand fusion

)(411

xe

Sigmoïde f

V Area 1

Area 2

Area 8

8

181 )/()()...(

iii VDPVPDDVP

Uniform)( VP

iDiiiiii

iiiiiiii DPDVPDP

DPDVPDPVDP

)/()/()(

)/()/()()/(

where :

Joint distribution for the fusion :

Probabilistic joint distribution for area i

[Pradalier et al. 03]

Tracking a given trajectory while avoiding sensed obstacles


Bayesian programming : some experimental resultsBayesian programming : some experimental results

Reactive trajectory tracking (Cycab) Target following (Koala)


V-Obstacles & ITP (dynamic environment)V-Obstacles & ITP (dynamic environment)

• Instantaneous escaping trajectories (V-obstacles) => Strategies for avoiding moving obstacles• Iterative Trajectory Planning => Complete navigation strategy

Relative VelocityCone (RVC)

)(ˆ)()(

)(ˆ)()(

)(

)()(:)()(

)(

tct

ritctvo

tct

ritctvo

etc

etdtctrajectoryObstacle

lvl

lvr

tittd

v

ti

Current work: More complex geometry & dynamics, Perception, Uncertainty

V-obstacles

V-obstacles+

ITP

[Large et al. 03]


Robust obstacles tracking Robust obstacles tracking (Bayesian Occupancy Filter approach)(Bayesian Occupancy Filter approach)

Occupiedspace

Freespace

Unobservable space

Concealed space

(“shadow” of the obstacle)

1 sensor & 1 object

2 sensors & 3 objects

z1,1 = (5.5, -4, 0, 0) z1,2 = (5.5, 1, 0, 0)z2,1 = (11, -1, 0, 0) z2,2 = (5.4, 1.1, 0,0)

P([Ec=1] | z1,1 z1,2 z2,1 z2,2 c)c = [x, y, 0, 0]

P( [Ec=1] | z c)c = [x, y, 0, 0] and z=(5,2,0,0)

Occupancy grid

– Variables :• C : cell• EC : cell occupancy (EC=1 means“occupied”)

• Z1:S : observations• M1:S : association (1 for each sensor)

– Parametric form :• P(EC C) : a priori uniform• P(Z1:S | EC C) : sensor models

Pro

gram

Pro

gram

Des

crip

tion

Des

crip

tion

Qu

esti

onQ

ues

tion

• UtilizationUtilization

• SpecificationSpecification

• IdentificationIdentification => Calibration

– Decomposition :

For all c : P(Ec | Z1:S c)

S

s

s

O

iSCis

S

ssC

SSC

MCEZPMPCEP

ZMECP

1,

1

:1:1

)( )()()(

)(


Robust obstacles tracking & avoidance Robust obstacles tracking & avoidance Experimental results with the CycabExperimental results with the Cycab

Prediction

Estimation

Bayesian Occupancy Filter (BOF)Bayesian Occupancy Filter (BOF)

Pedestrian avoidance using the BOFPedestrian avoidance using the BOF


Automatic reconstruction of a dynamic mapAutomatic reconstruction of a dynamic map(Parkview : Multi-camera system)(Parkview : Multi-camera system)

Dynamic Map

Static components Mobile components Changing components

Fusion

[Helin 03]


Trajectory prediction for moving obstaclesTrajectory prediction for moving obstacles

22

),(2

1

2

1)|(

kpp d

k

kp eCdP

2/1

0

2)()(1

),(

dttdtdT

ddpT

t

ipp

jpp

Learning phase : Learning phase : Clustering

kN

ii

kk td

Nt

1

)(1

)(

2/1

1

2),(1

kN

iki

kk d

N

Mean trajectory :

Standard deviation :

Prediction phase (at each time step) :Prediction phase (at each time step) :Calculating the likelihood thatCalculating the likelihood that kp Cd

Partial distance :

Likelihood estimation :

[Vasquez 04]


Some previous results ofSome previous results ofour research teamour research team

• Interactive medical simulationInteractive medical simulation

• Autonomous navigation for Virtual Reality Autonomous navigation for Virtual Reality applicationsapplications


Echographic simulatorEchographic simulator (c(coop. TimC & UC-Berkeley & LIRMM)[Daulignac & Laugier 00-01]

Stress-strain curves(measured)

)(

)(

linearnonbxa

xF

linearxkF

Geometric model

Measured data

+

Separating the Gall-bladderfrom the liver

[Boux & Laugier & Mendoza 01]

Cutting a flag using an haptic device[Boux & Laugier 00]

Interactive medical simulationInteractive medical simulation


• Dynamic path planning : Dynamic path planning : Adriane’s Clew Algorithm [Ahuactzin 94]

• Reactive navigation : Reactive navigation : => => Path tracking & Obstacle avoidance [Raulo &Laugier 00] => Bayesian behaviors [Lebeltel 99, Raulo 01]

Solving : P(Solving : P(Vrot Vtrans | | px0 px1 … … lm7 veille feu … … per))

Question : P( M | S Cp_Surveil)

._P....

SurveilCp

permldmnvdnv

vtrans_cproxGdirGproxdir tourtempo1-td_t1-tach_tengobj?feuveillelm7lm0px7px0

VtransVrot

Autonomous navigation for Virtual Reality Autonomous navigation for Virtual Reality applicationsapplications


Autonomous drivingAutonomous driving& &

Driving assistanceDriving assistance


The « CyberCars » approachThe « CyberCars » approach Door to door, 24 hours a dayDoor to door, 24 hours a day Small (urban size), silent Small (urban size), silent User friendly interface User friendly interface Automatic manoeuvresAutomatic manoeuvres => parking, platooning

… up to fully automated

CyberCars are focusing on historical city centres

Praxitele : Real experiment in SQY (97-99)Praxitele : Real experiment in SQY (97-99)

Industrial siteTrain station

Shopping centre

Tramway Metro

TGV

Bus

Private Car

Wal

k

FarNear

Low

High

Distance

Cap

aciy

....

> 500 m

BikeRoller

CyCab dual-mode vehicleCyCab dual-mode vehicleCommercialized by Robosoft

Private tracksLocal tracks

Pedestrian zonesCalm zones

Suburban tracks

Intercity tracksFull driving automation on :


Some CyberCars projectsSome CyberCars projects

ParkShuttle (Frog, Netherlands)ParkShuttle (Frog, Netherlands) Serpentine (Switzerland)Serpentine (Switzerland) PraxitelePraxitele ( (France)France)

European European CybercarsCybercars project (2001-05) project (2001-05) 10 industrial partners 10 industrial partners (Fiat, Yamaha, Frog …),(Fiat, Yamaha, Frog …), 7 research institutes 7 research institutes (Inria, Inrets,

Ensmp …), 12 cities involved , 12 cities involved (Rome, Rotterdam, Lausanne, Antibes …)(Rome, Rotterdam, Lausanne, Antibes …) 10 M €10 M €

E-Cab (Yamaha)E-Cab (Yamaha)CyCab (Robosoft)CyCab (Robosoft) ParkShuttle (Frog)ParkShuttle (Frog)

Serpentine (SSA)Serpentine (SSA)


The The « Automotive »« Automotive » approach approachADAS : Advanced Driver AssistanceADAS : Advanced Driver Assistance

European Prometheus project (86-94)European Prometheus project (86-94)

R&D programR&D program ( (on-board and off-board systemson-board and off-board systems) for increasing safety & driving confort) for increasing safety & driving confort

ACCStop&Go

Stop&Go ++

Rural Drive Ass.

Urban Drive Ass.

Full DrivingAutomation

Longitudinal ControlLongitudinal Control

+ Lateral Control+ Lateral Control

Current projects: Carsense, Arcos, PreventCurrent projects: Carsense, Arcos, Prevent

Carsense (car manufacturers & suppliers)Carsense (car manufacturers & suppliers)Sensor fusion for danger estimationSensor fusion for danger estimation

French Arcos project: French Arcos project: Vehicle-Infrastructure-Driver systems for road safetyVehicle-Infrastructure-Driver systems for road safety


Experimental vehicles at INRIA Rhône-AlpesExperimental vehicles at INRIA Rhône-Alpes

• electric vehicle with front driven and steering wheels• abilities of human or computer-driven motion• control system: VME CPU-board, transputer net• sensor : odometry, ultrasonic sensors, linear CCD camera

Ligier vehicleLigier vehicle

CycabCycab

Models for controlling the CycabModels for controlling the Cycab

22

2cos

BA

fLM

0 2

322

2 ''cos

uBuA

uBuAuAuBufLN

fffB

ffA

sincos'sincos

cos'cos 22

• Existence (Differential flatness) [ Sekhavat, Hermosillo, 99]

• Necessary conditions (« flat outputs ») [Sekhavat, Rouchon, Hermosillo 01]

• Analytic determination of the « flat outputs » [Sekhavat, Hermosillo, Rouchon 01]

• Motion planning & control (trajectory tracking) [Hermosillo 03] [Hermosillo & Pradalier 04]

)(

)()(1

cos'cos

tan)(),,(:

f

AB

ufuft

ttF withframeTurning

21

1

0

0

0

0cos

sinsin

cos

uu

L

ff

f

y

x

R

R

Differential Flatness of the CycabDifferential Flatness of the Cycab

Decisional & Control architectureDecisional & Control architecture

continuous curvature profile + upper-boundedcurvature & curvature derivative

continuous curvature profile + upper-boundedcurvature & curvature derivative

Kinodynamic Motion Planning(Dynamic constraints ...)

q

t

Graph

Trajectory

[Fraichard 92 ]Planning CC-paths

(kinematic constraints ...)

x

y

)(tan 1 w

w

[Scheuer & Laugier 98 ]

3-layered control architecture[Laugier et al. 98 ]

Decision module

Reactive mechanism=> Control the executionof the selected skills

Real-time “Skills” => Close-loop controls & Sensor processing

Platooning [Parent & Daviet 96] Automatic Parallel Parking [Paromtchik & Laugier 96]

Lane Changing & Obstacle avoidance [Laugier et al. 98]


« Platooning »« Platooning »

Electronic « Tow-bar »

CCD Linear camera + Infrared target(high rate & resolution)

[Parent & Daviet 96]


Automatic parking maneuversAutomatic parking maneuvers

Start location specification

[Paromtchik & Laugier 96]

( ) ( ),

( ) ( ),, , ,

( )

,

cos ,

,

, ,

( ) . cos ,

max

maxmax max

*

**

t k A t t T

v t v k B t t Tv k k

A t

t tt t

Tt t T t

T t t T

tT T

T T

B t t T

vv

,

0

00 0 1 1

1 0

12

05 1 4 0 t T

=> On-line motion planning using sinusoidal controls (t) and v(t)(search for control parameters T and max)

On-line local world reconstruction& Incremental motion planning

christian laugier – e-motion team-project 1 the e-motion team-project « geometry and probability...

Documents