Hybrid Manipulation: Force-Vision

CMPUT 610

Martin Jagersand

Today:

How to incorporate other sensory feedback.

Focus on tasks where the number of contact constraints increase during manipulations.

Acquire necessary constraint geometry on-line.

Preliminaries:Visual Servoing Applications

Limitations of Visual Control

Accuracy is limited by: Visual tracking and Visual goal specification

(Jagersand et. al. ICRA 97)

Specifying well defined visual encodings can be difficult (Hager, Hespana, Dodds, Jagersand)

Limited to non-occluded settings Not all tasks lend themselves to visual

specification.

Observation:Force constraints natural in tasks

Example:

Inserting a box Sequence of

motions Increasing number

of contact constraints

Preliminaries: Uncalibrated Visual Servoing

Let y = visual observation; x = motor control Linear system model:

y= Linear p-controller

Estimate the Visual-Motor Jacobian

Jêk+1 = Jêk+ É xTÉ x

(É ymeasuredà JêkÉ x)É xT

f (x) ù f (xk) + J (xk)(x à xk)

xç = K J +kyç

What do we need to add to do things like this?

Learning Constraint Geometry

Impact force along surface normal:

Sliding motion:

3rd vector:

p1 = jfkjfk

p2 = jî xkjî xk

p3 = p1â p2 = jfkjfk â

jî xkjî xk

Constraint Frame

With force frame = tool frame we get:

Assume frictionless => Can update each time step

Pk+1 =p1p2p3

0

@

1

A =

jfkjfk

jî xkjî xk

jfkjfk â

jî xkjî xk

0

BBB@

1

CCCA

P1

P2

P3

Hybrid Control Law

Let Q Joint -> Tool Jacobian Let S be a switching matrix, e.g. diag([0,1,1]) Velocity control u:

uk = à K vQà1Pà1kSPkQJ à1ek à K cQà1Pà1

k(I à S)Pkfk

Visual part Force part

Accounting for Friction

Friction force is along motion direction! Subtract out to recover surface normal:

Pk+1 =

jfkàpT

2fkp2j

fkàpT

2fkp2

jî xkjî xk

jfkàpT

2fkp2j

fkàpT

2fkp2

âjî xkjî xk

0

BBBBBB@

1

CCCCCCA

Final Hybrid Controller Structure

Motion Sequence

Switching of Constraints inHybrid Force-Vision Feedback

Sliding along a Curved Surface

In principle any smooth surface can be followed

Detail of measured forceEstimated surface normals

Estimation Accuracy

The constraint coordinate frame was acquired within a few degrees of the true surface frame

Conclusions

Hybrid Force-Vision control allow more accurate positioning than vision or force alone

Many tasks naturally involve an increasing number of contact constraints

We showed that the contact geometry can be successfully estimated

Combined Uncalibrated Vision & Force control