seoul national university bio-inspired robot motor learning f.c. park seoul national university
TRANSCRIPT
SEOUL NATIONAL UNIVERSITY
Bio-inspired Robot Motor Learning
F.C. ParkSeoul National University
SEOUL NATIONAL UNIVERSITY
The State-of-the-Art
Sony Q-Rio Demonstration
SEOUL NATIONAL UNIVERSITY
Some Questions
Is there a “physical intelligence” behind these movements, e.g., are they based on certain “primitives” that can be reused and generalized to new contexts (learning)? Or are they just manually created movements?
How much “feedforward” versus “feedback” is used in the motor program? What, if any, distinctions can be made between “planning” and “control”?
SEOUL NATIONAL UNIVERSITY
Lessons from Human Motor Control
Bernstein’s degrees of freedom problem and motor synergies
Movement learning as a transition process from closed-loop to open-loop control.
Alain Berthoz’s theories about perception as simulated action—the brain as a “simulator” that anticipates (Alain Berthoz,The Brain’s Sense of Movement)
SEOUL NATIONAL UNIVERSITY
Topics of this (Very Brief) Talk
Describe my attempts to build a general framework for movement generation
Creating feedforward plans: dynamically optimal movements.
Movement primitives using principal components and HMMs
Primitives for balancing and posture
SEOUL NATIONAL UNIVERSITY
State-of-the-Art in Robotics
Operational space control and elastic roadmaps (Khatib, Brock): Potential functions in operational space are used to define goal “attractors”, avoid collisions, etc., all the while exploiting null space dynamics to, e.g., maintain posture.
Rapidly-exploring random trees (RRT) (Kuffner, LaValle): Incrementally growing a randomized search tree by applying control inputs over short intervals.
SEOUL NATIONAL UNIVERSITY
Elastic Roadmap Implementation
SEOUL NATIONAL UNIVERSITY
Is There Room for Improvement?
Potential functions only capture feedback; little if any feedforward or memory-based aspects of movement.
Potential functions for, e.g., dynamic balance, are not easily formulated (ZMP depends on COM acceleration coordinates)
RRT methods are sensitive to the choice of metric, and nearest neighbor computations are still a burden
The resulting motions are not necessarily optimal or reusable.
SEOUL NATIONAL UNIVERSITY
Creating Optimal Movements
Minimize an integral cost functional of the form
subject to the dynamic equations
and various boundary conditions and
constraints on the state and control
SEOUL NATIONAL UNIVERSITY
Example: Minimum Torque Motions
Numerical algorithms like steepest descent, Newton-type methods require gradient (and sometimes Hessian) information:
Dynamic equations must be differentiated.
Without analytic gradients (and sometimes Hessians), convergence is unreliable.
SEOUL NATIONAL UNIVERSITY
Robot Dynamics Equations Dynamics for a standard 6-axis industrial robot:
SEOUL NATIONAL UNIVERSITY
Robot Dynamics Equations (p2)
SEOUL NATIONAL UNIVERSITY
Robot Dynamics Equations (p3)
SEOUL NATIONAL UNIVERSITY
Robot Weightlifting Demo (w/J. Bobrow)
PUMA Robot Weightlifting Demo
SEOUL NATIONAL UNIVERSITY
Why Can’t Robots Jump like Humans?
The human biarticular muscle is important in the generation of fast explosive human movements.
Spring-like behavior (energy restoring and release) is achieved by the elastic property of muscle, and efficient energy transfer by extension over two joints improves, e.g., jumping performance.
Biarticular structures for robots are intriguing.
< monoarticular muscle > < biarticular muscle >
SEOUL NATIONAL UNIVERSITY
A Biarticular Robot Leg (J. Babic)
< Robot model >
(1) Initially acts as passive prismatic joint
(2) When biarticular mechanism is activated, robot becomes redundantly actuated by biarticular force
(3) Immediately after robot pushes off the ground, the biarticular actuator is deactivated
0
0
0
-
-
-
( )biarticular
x b a
x b a
b a
k
f k x x
Length of
Length of at the moment of activation
Vector representing the direction of force
Biarticular link stiffness (N/m)
Activation moment angle
SEOUL NATIONAL UNIVERSITY
Optimal Jumps for Biarticular Legs Dynamics model involves not only a closed chain with redundant
actuation and spring element, but also contact models between the feet and ground.
Lemke’s algorithm for the LCP formulation to solve contact problem with friction
Time stepping schemes to reduce integration error
Optimization involves diverse parameters in addition to the actuator inputs: Spring stiffness, Biarticular actuator activation angle
,
,
( ) ( , ) a
p
c b
c b
a aT Tc c b b
p p
q
q
F F
J J
qM q b q q J F J F
q
active joints
passive joints
Generalized contact forces and biarticular forces
J acobians for each position and orientation
SEOUL NATIONAL UNIVERSITY
Maximum Height Jumps Maximum height vertical jump
Nearly 23% higher (comparing the conventional robot)
Dynamics constraints considered. Joint torque limits, joint velocity limit
Conventional robot
< Initial motion >
Biarticular legged robot (k=10000N/m, )
< Opimized motion >0
90
SEOUL NATIONAL UNIVERSITY
Maximum Distance Broad Jumps Maximum distance broad jump
Nearly 20% longer (comparing the conventional robot)
Dynamics constraints considered. Joint torque limits, joint velocity limit
Conventional robot
< Initial motion >
Biarticular legged robot (k=5000N/m, )
< Optimized motion > 080
SEOUL NATIONAL UNIVERSITY
Optimization: Some Shortcomings
Computationally expensive, not real-time.
Results are not readily re-usable. Not clear how to, e.g., perform complex
movement sequences, or optimally perform multiple tasks.
SEOUL NATIONAL UNIVERSITY
Reusability: Movement Primitives
Movement primitives for reaching motions are extracted by a principal component analysis of training data (e.g., dynamically optimal movements, motion capture data)
The primitives can serve as basis functions for linear interpolation.
The primitives can also serve as basis functions for dynamic optimization.
SEOUL NATIONAL UNIVERSITY
PCA of Arm-Reaching Motions
Repeated trials of an arm motion (e.g., lifting) are performed
PCA
First 4 principal components for each jointJoint angle trajectories of sample motions
SEOUL NATIONAL UNIVERSITY
Sample Primitive Database
TaskTrial
s
raise hand 156
put down hand 156
reach out hand 88
spread arms 89
revolve arms forward 55
revolve arms backward 60
raise hand with dumbbell
100
put down hand with dumbbell
100
hold out hand for handshaking
100
handshaking 100
restore hand after handshaking
100
small hand-waving 92
big hand-waving 88
bow 54
SEOUL NATIONAL UNIVERSITY
Example Reaching Movements
Optimal Movements Sub-Optimal Movements
Optimal movements actually appear less natural than the PCA-based (suboptimal) movements
SEOUL NATIONAL UNIVERSITY
Primitives for Balancing & Posture Given:
A legged robot standing on some subset of its legs
An input reference joint trajectory, possibly dynamically unstable
Objective: Adjust the reference joint trajectory in real-time to maintain posture while tracking the input motion withstanding external
disturbances
SEOUL NATIONAL UNIVERSITY
An Optimization Formulation
General balancing can be cast as a second-order cone programming (SOCP) problem:
nfzZMPfzZMP
MzZMPMzZMP
fxyZMPfxyZMP
COMCOM
ViconstV
MxyZMPMxyZMP
ref
ttCxM
tCxM
tCxM
bxA
bVxA
CxM
txx
t
ii
21,,
2,,
1,,
,
,,
0
0
0
subject to
min
SEOUL NATIONAL UNIVERSITY
A Hierarchy of Optimization Problems
A partial hierarchy of convex optimization problems:
SOCP is a slightly generalized version of QCQP
General formulation of SOCP problem (c = 0 leads to QCQP):
General features of convex programming problems: Any local optimum is also a global optimum. There exist many algorithms and software, e.g., MOSEK, free software
by Boyd and Vandenberghe, etc.
gFx
idxcbxA
xf
iTiii
T
subject to
min
LCLP LCQP QCQP SOCP NLCP⊂ ⊂ ⊂⊂
SEOUL NATIONAL UNIVERSITY
Maintaining Posture in Real-Time
The original motion is a standing posture. The results closely resemble a human’s reaction.
Acceleration profile
Time/step
(along x-axis)
1.4 0.423052
-0.9 0.390567
(rotation about y-axis)
6 0.385343
-6 0.398187
sina ta
2 sina t
2 / ft The units of are and a 2/m s 2deg/ s
SEOUL NATIONAL UNIVERSITY
Kicking while Maintaining Balance
Acceleration profile
Time/step
(along x-axis)0.7 0.38905
2
-0.8 0.392851
(along y-axis)0.45 0.38260
4
-0.65 0.395679
(rotation about y-axis)4.5 0.39582
8
-2.5 0.385179
(rotation about x-axis)3 0.38397
0
-3 0.391993
sina t
a
2 sina t
sina t
2 sina t
Original (unstable) kick
Stabilized kick
ZMP trajectories COG trajectories
SEOUL NATIONAL UNIVERSITY
Towards a Unified Motor Control Theory
Further inspiration from human motor control: Minimum Variance Principle (Wolpert),
Minimum Intervention Principle (Todorov and Jordan)
The Minimum Attention Paradigm (Brockett): Often imprecise control is as good as precise control—
reducing control implementation costs is more important.
Defining control implementation costs: the easiest control is a constant control.
One possible optimal control formulation:
),(
),,(
subject to
min
uxfdt
dx
dxdtdx
du
dt
duxL