outline im-clever: intrinsically motivated cumulative learning versatile robots
DESCRIPTION
IM-CLeVeR: Intrinsically Motivated Cumulative Learning Versatile Robots Gianluca Baldassarre , Marco Mirolli , Francesco Mannella, Vincenzo Fiore, Stefano Zappacosta, Daniele Caligiore, Fabian Chersi, Vieri Santucci, Simona Bosco. - PowerPoint PPT PresentationTRANSCRIPT
Gianluca Baldassarre 1/20Sestri Levante, 18 January 2010
IM-CLeVeR: Intrinsically MotivatedCumulative Learning
Versatile Robots
Gianluca Baldassarre, Marco Mirolli,Francesco Mannella, Vincenzo Fiore, Stefano Zappacosta,
Daniele Caligiore, Fabian Chersi, Vieri Santucci, Simona Bosco
Gianluca Baldassarre 2/20Sestri Levante, 18 January 2010
OutlineIM-CLeVeR: Intrinsically Motivated
Cumulative Learning Versatile Robots
The “numbers” of the project The partners The project vision The 3 pillars of the project hypothesis + 4 S/T objectives WP3: Experiments WP4: Abstraction WP5: Intrinsic motivations WP6: Hierarchical architectures WP7: Integration and demonstrators Conclusions
Gianluca Baldassarre 3/20Sestri Levante, 18 January 2010
The “Numbers” of the ProjectIM-CLeVeR: Intrinsically Motivated
Cumulative Learning Versatile Robots
Integrated Project Coordinator: ISTC-CNR Call: Cognitive Systems, Interactions and Robotics EU funds: 5.9 ml euros 7 Partners Start: May 2009 End: April 2013
Gianluca Baldassarre 4/20Sestri Levante, 18 January 2010
Partners1. CNR-ISTC-LOCEN Coordinator
(Gianluca Baldassarre, Marco Mirolli)
1. CNR-ISTC-UCP (Elisabetta Visalberghi)
1. UMASS (Andrew Barto)
2. USFD (Peter Redgrave & Kevin Gurney)
3. UCBM-LBRB (Eugenio Guglielmelli)
3. UCBM-LDN (Flavio Keller)
4. FIAS (Jochen Triesch)
5. AU (Mark Lee)
6. UU (Ulrich Nehmzow)
7. IDSIA (Juergen Schmidhuber)
Institutes Groups
Gianluca Baldassarre 5/20Sestri Levante, 18 January 2010
Vision: the problem How can we create “truly intelligent” robots?
Versatile: have many goals; re-use actions Robust: function in different conditions, with noise Autonomous: learning is paramount
Weng, McClelland, Pentland, Sporns, Stockman, Sur, Thelen, (Science, 2001):
…knowledge-based systems (e.g. production systems)… …learning systems focussed on single tasks (e.g. RL)… …evolutionary systems… Important results, but limited autonomy and scalability. . . . . . on the contrary . . .
. . . organisms do scale, are flexible, and are robust!
Gianluca Baldassarre 6/20Sestri Levante, 18 January 2010
Vision: the idea Why are organisms so special? Let’s give a closer look at children…
Gianluca Baldassarre 7/20Sestri Levante, 18 January 2010
Vision: the ideaIngredients: Powerful abstractions: “elefant on table leg”, “it slides down” Explore and record interesting states:
Based on intrinsic motivations (novelty, learning rates, …) Such states motivate to reach them (= goals) Furnish learning signals which guide learning
Acquired skills are: Re-used to explore and discover new goals Composed to produce new skills
Science: which brain and behavioural mechanisms are behind these processes?
Technology: Can we reverse engineer them?
Gianluca Baldassarre 8/20Sestri Levante, 18 January 2010
Vision: 2 promises Science: we can understand the mechanisms in organisms Technology: we can develop a new methodology for designing robots…
… in particular robots that (we will get 3 iCubs!)…
Learn actions cumulatively:
…on the basis of abstraction
(sensory andmotor)…
…on the basis of intrinsic
motivations…
…on the basis of already learned
actions.
Gianluca Baldassarre 9/20Sestri Levante, 18 January 2010
Vision: how we will do it:3 pillars + 4 S/T objectives
WP4: Abstraction and attention
WP5: Intrinsic motivations
WP6: Hierarchical architectures to support
cumulative learning
1. Empirical investigations:
- Monkeys - Children - Adults - Parkinson patients
4. Two robotic demonstrators:- CLEVER-B- CLEVER-K
2. Computational bio-constrained models:mechanisms underlying brainand behaviour
Suitable representations
Focussing learning
Science
From Science to Technology
Technology
3. Machine-learning models:powerful algorithms and architectures
From Technologyto Science
Gianluca Baldassarre 10/20Sestri Levante, 18 January 2010
WP3: Experiments and mechatronic board
WP4: Abstraction and attention
WP5: Intrinsic motivations
WP6: Hierarchical architectures to support
cumulative learning
1. Empirical investigations:
- Monkeys - Children - Adults - Parkinson patients
4. Two robotic demonstrators:- CLEVER-B- CLEVER-K
2. Computational bio-constrained models:mechanisms underlying brainand behaviour
Suitable representations
Focussing learning
Science
From Science to Technology
Technology
3. Machine-learning models:powerful algorithms and architectures
WP3
Gianluca Baldassarre 11/20Sestri Levante, 18 January 2010
WP3: Empirical Experiments: “Board experiment” UCBM-LBRB (Eugenio Guglielmelli); UCBM-LDN (Flavio Keller): children
CNR-ISTC-UCP (Elisabetta Visalberghi): monkeys;
Inertial/magnetic unit + battery + wireless
Tactile sensors
Sabbatini, Stammati, Tavares, Visalberghi, 2007,Amer. J. PrimatologyCampolo, Taffoni, Schiavone,
Formica, Guglielmelli, Keller, 2009, Int. J. Sicial Robotics
Gianluca Baldassarre 12/20Sestri Levante, 18 January 2010
WP4: Abstraction
WP4: Abstraction and attention
WP5: Intrinsic motivations
WP6: Hierarchical architectures to support
cumulative learning
1. Empirical investigations:
- Monkeys - Children - Adults - Parkinson patients
4. Two robotic demonstrators:- CLEVER-B- CLEVER-K
2. Computational bio-contrained models:mechanisms underlying brainand behaviour
Suitable representations
Focussing learning
Science
From Science to Technology
Technology
3. Machine-learning models:powerful algorithms and architectures
WP4
Gianluca Baldassarre 13/20Sestri Levante, 18 January 2010
WP4 Abstraction: motor, perception, attention, vergence,
Abstraction is a key ingredient for action hierarchies Abstraction is a key ingredient for intrinsic motivations
Schembri, Mirolli, Baldassare, 2007,ICDL, ECAL, EPIROB
Neto Nehmzow, 2007, Rob. & Aut. Syst.
Gianluca Baldassarre 14/20Sestri Levante, 18 January 2010
WP5: Novelty detection
WP4: Abstraction and attention
WP5: Intrinsic motivations
WP6: Hierarchical architectures to support
cumulative learning
1. Empirical investigations:
- Monkeys - Children - Adults - Parkinson patients
4. Two robotic demonstrators:- CLEVER-B- CLEVER-K
2. Computational bio-contrained models:mechanisms underlying brainand behaviour
Suitable representations
Focussing learning
Science
From Science to Technology
Technology
3. Machine-learning models:powerful algorithms and architectures
WP5
Gianluca Baldassarre 15/20Sestri Levante, 18 January 2010
WP5 Intrinsic (extrinsic) motivations Extrinsic motivations
(e.g. food, sex, money): Psychology (Berlyne,
White, Deci & Rayan):motivate actions to achieve specific goals
Drive actions whose effects directly increase fitness
Come back again with the homeostatic needs they are associated with
Intrinsic motivations (skill/knowledge acquis.):
Psychology: motivate actions for their own sake
Drive actions whose effects are an increase in:(a) knowledge or prediction ability;(b) competence to do
Terminate to drive actions when knowledge or competence is acquired
Gianluca Baldassarre 16/20Sestri Levante, 18 January 2010
WP5 Intrinsic motivations CNR-LOCEN (Gianluca Baldassarre, Marco Mirolli) Young robot: low level of hierarchy develps skills based on
evolved ‘reinforcers’ (knowledge-based intrinsic motivations) Young robot: high level of hierarchy selects skills which produce
the highest suprise (competence-based intrinsic motivations) Adult robot: high level of hierarchy performs skill composition to
achieve salient goals (external rewards fitness measure)Adult robot tasks
Child robot taskYoung robot: resultsBefore learning
After learning
Adult robot: results
Schembri, Mirolli, Baldassare, 2007, ICDL, ECAL, EPIROB
Gianluca Baldassarre 17/20Sestri Levante, 18 January 2010
WP6: Hierarchical architectures
WP4: Abstraction and attention
WP5: Intrinsic motivations
WP6: Hierarchical architectures to support
cumulative learning
1. Empirical investigations:
- Monkeys - Children - Adults - Parkinson patients
4. Two robotic demonstrators:- CLEVER-B- CLEVER-K
2. Computational bio-mimetic models:mechanisms underlying brainand behaviour
Suitable representations
Focussing learning
Science
From Science to Technology
Technology
3. Machine-learning models:powerful algorithms and architectures
WP6
Gianluca Baldassarre 18/20Sestri Levante, 18 January 2010
WP6 Bio-inspired / bio-constrained hierarchical reinforcement learning
CNR-LOCEN (Gianluca Baldassarre & Marco Mirolli) Piaget theory: actions support learning of other actions Camera, dynamic arm, reaching tasks Continuous state/action reinforcement learning Hierarchical RL: segmentation, Piaget
Caligiore Borghi Parisi Mirolli Baldassarre, ongoing From Fuster, 2001, Neuron
Gianluca Baldassarre 19/20Sestri Levante, 18 January 2010
Conclusions: A timely project! Timely research goals:
sensorimot. abstraction, intrinsic motiv., hierarchical architect. Within important trends:
Developmental robotics Computational system neuroscience Emotions/motivations
In synergy with various events:EpiRob, ICDL, IEEE Journal Automonous Mental Development
In line with EU calls:“Cognitive Systems, Interactions and Robotics”
First EU Integrated Project wholly focussed on these topics
www.im-clever.eu
Gianluca Baldassarre 20/20Sestri Levante, 18 January 2010
What we need from iCub
Robustness! Usability (assistance):
Software: Yarp, simulator for rapid prototyping Hardware: when it will break
One standardised simulator (e.g., based on Bullet) Compliance: for safety, for more bio-realism That it actually becomes a standard in EU research