Teaching Robots to

Drink, Relax and Play catch

Factory Robots

Factory Robots

Factory robots live in a human-less world.

Factory robots function in a world that is fully predetermined,

where there is no room for change.

Factory Robots

Going into the real world

Unlike industrial settings, the world

in which we live changes all the

time:

Cannot predict all circumstances

Need to react rapidly and

appropriately

Robots that deal with uncertainty

Commercial airplanes fly autonomously to a very large extent.

They can recover from turbulences automatically.

Automobile industry and governments support research

to build fully autonomous cars.

Robots that deal with uncertainty

No two apartments

look the same

And the same apartment

can change appearance

from one day to the next

Uncertainty in home environment

What does it mean to grate carrots?

More than one way to do this.

More than one tool to

perform the task.

Variability in task definition

Learning from Human Demonstrations

Learning a skill is more than simply replaying a trajectory.

It requires to understand what a skill is.

To learn this, one needs to show several demonstrations

to generalize across sets of examples.

Learning from Human Demonstrations

Learning a skill is more than simply replaying a trajectory.

It requires to understand what a skill is.

To learn this, one needs to show several demonstrations

to generalize across sets of examples.

Learning from Human Demonstrations

K. Kronander, M. Khansari and A. Billard, JTSC Best Paper Award, Int. Conf. on Intelligent and Robotics Systems, IROS 2011

Learning from Human Demonstrations

Teaching robot how to adapt to perturbations

http://lasa.epfl.ch

Being stiff is not always good How to teach a robot to relax…

Teaching robots to be less stiff

Low stiffness when carrying the liquid High stiffness when pouring the liquid

http://lasa.epfl.ch

Shaking the robot: A natural method to teach a robot to relax.

Teaching robots to be less stiff

Being stiff is not always good How to teach a robot to relax…

http://lasa.epfl.ch

After training the robot manages to adapt naturally when

required and remains stiff when required.

Teaching robots to be less stiff

Learning from Failure

Learning from Failure

Training examples

The robot is provided solely with failed examples.

It has no information about the task – no reward, no indication of

what was incorrect.

D. Grollman and A. Billard, Best Cognitive Robotics Paper Award, Int. Conf. on Robotics and Automation, ICRA 2011

Reproduction

Learning from Failure

Find a solution in a few trials

Is comparable in efficiency to classical reinforcement learning approaches

But does not need a reward!

http://lasa.epfl.ch

Teaching Robots to be Highly Reactive

http://lasa.epfl.ch

Learning a control law that ensures that you reach the target even if perturbed

and that you follow a particular dynamics

Generalizing: Learning a control law

http://lasa.epfl.ch

Coupled Dynamical Systems

Decoupled control of hand and fingers may lead to

failure when adapting to very rapid perturbations.

Coupled control of hand and fingers ensures that fingers and

hand close in a coordinated manner on the new target.

http://lasa.epfl.ch

Adaptation to perturbation of the order of a few millisecunds.

Coupled Dynamical Systems Coupled Dynamical Systems for Reach and Grasp

http://lasa.epfl.ch

Catching Objects in Flight

http://lasa.epfl.ch

Catching Objects in Flight

Extremely fast computation (object flies in half a second); re-estimation of

arm motion to adapt to noisy visual detection of object.

http://lasa.epfl.ch

STEP 1: Build a model of the graspable region on the object;

Catching Objects in Flight

http://lasa.epfl.ch

STEP 1: Build a model of the graspable region on the object; learn likelihood

of placing fingers in region of the handle from several demonstrations;

X (m)

Z (

m)

-0.05 -0.04 -0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

Z (m

)

X (m)

Catching Objects in Flight

x

z

http://lasa.epfl.ch

STEP 1: Build a model of the graspable region on the object; learn likelihood

of placing fingers in region of the handle from several demonstrations;

X (m)

Z (

m)

-0.05 -0.04 -0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05

-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

x

z

y

X (m)

Y (

m)

-0.05 -0.04 -0.03 -0.02 -0.01 0 0.01 0.02 0.03 0.04 0.05-0.04

-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

X (m)

z

Y (m

)

x

Catching Objects in Flight

y x

z

Z (m

)

X (m)

http://lasa.epfl.ch

If the motion of the object is complex, a simple

ballistic model is not sufficient

Needs to estimate the dynamics of object in flight

Catching Objects in Flight

STEP2: Learn a model of the translational and rotational motion of the object

http://lasa.epfl.ch

STEP2: Gather several examples

Catching Objects in Flight

Use non-linear regression model (Support

Vector Regression)

Precision (1cm, 1degree); computation

0.17-0.32 second ahead of time.

Combine with extended Kalman Filter to

tackle innacuracy of vision.

S. Kim and A. Billard, Aut. Robots, 2012

http://lasa.epfl.ch

STEP 3: Compute the region of feasible

hand postures that yield a possible grasp.

Catching Objects in Flight

http://lasa.epfl.ch

STEP 3: Compute the region of feasible hand postures that yield a possible

grasp through sampling space.

X (m)

Y (

m)

-0.6 -0.4 -0.2 0 0.2 0.4

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

X (m)

Z (

m)

-0.8 -0.6 -0.4 -0.2 0 0.2 0.4

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

Y (m)

Z (

m)

-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6

0

0.1

0.2

0.3

0.4

0.5

sX

sY

-5 -4 -3 -2 -1 0 1 2 3 4

-3

-2

-1

0

1

2

3

sX

sZ

-5 -4 -3 -2 -1 0 1 2 3 4

-3

-2

-1

0

1

2

3

sY

sZ

-6 -5 -4 -3 -2 -1 0 1 2 3

-3

-2

-1

0

1

2

3

Probability Contour

sX

sZ

-1.5 -1 -0.5 0 0.5 1 1.5

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Position Orientation

Catching Objects in Flight

http://lasa.epfl.ch

STEP 4: Find the grasping posture by

predicting dynamics of motion and finding

most likely combination of grasping point

and feasible hand posture.

Catching Objects in Flight

http://lasa.epfl.ch

Catching Objects in Flight

STEP 5: Generate motion of hand and fingers

to catch the object at the right place using

coupled dynamical systems for hand position

and orientation and for finger motion.

http://lasa.epfl.ch

Catching a flying object

Kim, Shukla and Billard: In preparation

Catching Objects in Flight

http://lasa.epfl.ch

Catching a flying object

Kim, Shukla and Billard: In preparation

Catching Objects in Flight

Our funny robots

The lab – Class of 2011

The lab – Class of 2012

http://lasa.epfl.ch

Sponsors

Thanks to the lab – Class of 2011

Photo by Lucia Pais & Basilio Noris