cognitive systems in the fp6 ist programme ist 2004, the ...holk cruse university of bielefeld...

Cognitive Systems in the FP6 IST Programme IST 2004, The Hague, 16th November 2004


CoSyCognitive Systems for Cognitive Assistants

Henrik I ChristensenKungl Tekniska Högskolan

J. Kevin O’ReganCNRS, University of Paris V

Jeremy Watts & Aaron SlomanUniversity of Birmingham

Ales LeonardisUniversity of Ljubljana

Bernt SchieleTechnical University of Darmstadt

Hans Uszkoreit & Geert-Jan KruijffDFKI

Bernhard Nebel & Wolfram BurgardAlfred Ludvig University of Freiburg

www.CognitiveSystems.org


GOALAdvance the Science of Mind

TARGET DOMAINHuman-like autonomous physical (robotic) systems

e.g. machines to help house-bound people who can’t rely on others

FOCUSIntegration of capabilities- perception- action- representations - interaction- reflection ...

PRACTICAL VALIDATIONExplorer robot

Playmate robotPhilosopher agent


CORE IDEASelf-modifying architecture(reactive, deliberative, reflexive)

CORE IDEACombine many capabilities:

Deployed concurrentlyInteracting asynchronouslyDeveloping over time

CRITICAL ISSUESLife-long open-ended incremental learningBalance between innate and acquired knowledgeCommunication: language & representations

CRITICAL ISSUESMultiplicity of representations

Recognition of affordancesCoping with uncertainty

Introspection


CoSyCognitive Systems for Cognitive Assistants

Develop the Fundamentals of Cognitive SystemsAsk and Answer the Big Questions!

Think: Integration

www.CognitiveSystems.org


JASTJoint-Action Science and Technology

Harold BekkeringInstitute for Cognition & Information

Ivan ToniCentre for Cognitive Neuroimaging

Radboud University Nijmegen(KUN)

Alois KnollTechnical University of Munich

Nikolaos UzunogluInstitute of Communication &

Computer Systems, Athens

Jan-Peter de RuiterMax Planck Institute for PsycholinguisticsNijmegen

Ellen Gurman BardUniversity of Edinburgh

Astros ChatziastrosMax Planck Institute for Biological CyberneticsTubingen

Estela BichoUniversidade do Minho

http://212.205.85.69/jast/index.htm


GOALExtend cognitive systems to multi-agent scenarios

PRACTICAL VALIDATIONTwo autonomous robotic agents co-operating on a

complex construction task

TARGET DOMAINRobot-robot and human-robot co-operative joint action

FOCUSExploitation of shared beliefs, intentions, and goals

Joint-Action

Science and TechnologyScience and Technology


CORE IDEADialogue supported perception, reasoning, & action: cross-modal associations

CORE IDEASelf-organizing systems(non-linear dynamical systems model of cognition)

Inspired by brain imaging data

CRITICAL ISSUESDetection and repair of

performance errors (derived from both self and partner)

CRITICAL ISSUESImitationReal-time adaptive planningVerbal and non-verbal communication

Joint-Action

Science and TechnologyScience and Technology


JASTJoint-Action Science and Technology

Co-operating Cognitive Systems Dynamical Models

Think: Shared Understanding

http://212.205.85.69/jast/index.htm


RobotCubRobotic Open-Architecture for

Cognition, Understanding, and BehavioursKerstin Dautenhahn

University of HertfordshireGiulio SandiniDIST, University of Genova

Paolo DarioScuola Superiore Sant’Anna, Pisa

Darwin CaldwellUniversity of Salford

Jose Santos VictorInstituto Superior Tecnico, Lisbon

Rolf PfeifferUniversity of Zurich

Auke BillardEcole Polytechnique Federale de Lausanne

Claes von HofstenUniversity of Uppsala

Francesco BecchiTeleRobot S.r.l.

Luciano FadigaUniversity of Ferrara

Emilio BizziEuropean Brain Research Institute

www.RobotCub.org



Cognition, Understanding, and Behaviours

Rodney BrooksMIT

Gordon ChengATR

Juergen KonczakUniversity of Minnesota

Yasuo KuniyoshiUniversity of Tokyo

Hideki KozimaNICT (CRL)

www.RobotCub.org


GOAL54 degree of freedom cognitive humanoid robot

TARGET DOMAINOpen research platform: humanoid robot size & shape of a 2 year old child

PRACTICAL VALIDATIONHand-eye coordinationGrasp and manipulate

lightweight objectsCrawl on all fours and sit up

FOCUSCo-development of cognitive skills through exploration and manipulation


CORE IDEACognition emerges through exploratory learning and co-developmentthrough embodied physical interaction

CORE IDEAInitially deals with immediate events

Increasingly acquires a predictive capability

CRITICAL ISSUESExplorationManipulationImitationGestural Communication

CRITICAL ISSUESPhylogenic configuration vs

Ontogenic development

tAnticipation / Planning / Deliberation / Prediction



Cognition, Understanding, and Behaviours

Cognition through Co-developmentExploration, Manipulation, Gesture, & Imitation

Think: Open Cognitive Humanoid Systems

www.RobotCub.org


COSPALCognitive Systems using Perception-Action

Learning

Gösta GranlundLinköping University

Gerald SommerChristian-Albrechts-University of Kiel

Josef KittlerUniversity of Surrey

Vaclav HlavacCzech Technical University in Prague

www.cospal.org


GOALNew architecture that combinesSymbolic Reasoning & Artificial Neural Networks

PRACTICAL VALIDATIONEmulate capability of

small children to solve manipulation puzzles

TARGET DOMAINRobotic exploration, Manipulation, & Problem solving

FOCUSSystem architecture able to semi-autonomously & incrementally build up knowledge and solution strategies


CORE IDEAAction precedes perception:Associations of perception and action are triggered by the system’s own actions

CORE IDEA

Bi-directional interface between- Perception-Action module- Symbolic module

CRITICAL ISSUESRobustnessGeneralizabilityScalability

CRITICAL ISSUESInitial bootstrappingIncremental learning

AttentionExploration and adaptation


COSPALCognitive Systems using Perception-Action Learning

Combination of Symbolic Reasoning & Neural Networks

Think: Action-dependent Perception

www.cospal.org


MACSMulti-Sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances

Erich RomeFraunhofer AIS

Erol SahinMiddle Eastern Technical University, Ankara

George DorffnerÖFAI, Vienna

Lucas PalettaJoanneum Research

Patrick DohertyLinköping University

www.macs-eu.org


GOALNew architecture for mobilegoal-directed autonomous manipulation

PRACTICAL VALIDATIONMobile manipulator robot

using vision, 3-D laser scanner, 2 DoF arm & magnetic gripper

TARGET DOMAINMobile manipulation robotics

FOCUSAffordance: ‘a resource or support that the environment offers an agent for action’, which the agent can directly perceive and employ(e.g. drivability of a track, graspability of an object)


CORE IDEAUse affordance to couple perception, action, and reasoning

CORE IDEAPerception of affordancesimplies interpretation based on capabilities for physical action and current goals

Acquired by learning

CRITICAL ISSUESControl based on

action-oriented interpretation(affordance-based control)

CRITICAL ISSUESArchitecturePerceptionRepresentationLearning


MACSMulti-Sensory Autonomous Cognitive Systems Interacting with Dynamic Environments for Perceiving and Using Affordances

Affordance-based integration of perception, action, and reasoning

Think: Affordances

www.macs-eu.org


SPARKSpatio-temporal Patterns for

Action-oriented Perception in Roving Robots

Paolo ArenaUniversity of Catania

Ángel Rodrígues VázquezANAFOCUS

Holk CruseUniversity of Bielefeld

Manuel VelardeIP-UCM, Universidad Computense, Madrid

Ákos ZarándyANALOGIC

Barbara WebbUniversity of Edinburgh

www.spark.diees.unict.it


GOALNew biologically-inspired architecture for action-oriented perception

TARGET DOMAINMobile robotics

FOCUSNon-linear dynamical systems theoryMulti-modal sensors

PRACTICAL VALIDATIONRobot guidance system


CORE IDEAPerception is a spatio-temporal pattern formation process which directly influences the associated motor behaviour

CORE IDEADynamically emergent patterns

Spatio-temporal array computer (SPARC)

CRITICAL ISSUESSynergeticsNon-linear dynamical systems theoryEmergent computationDynamic pattern formation

CRITICAL ISSUESIntegration of sensor stimuli

Concise iconic representation


SPARKSpatio-temporal Patterns for

Action-oriented Perception in Roving Robots

Perception-Action asEmergent Spatio-Temporal Patterns

Spatio-temporal Array Computer (SPARC)

Think: Non-Linear Dynamical Systems

www.spark.diees.unict.it


Mind RACESfrom Reactive to Anticipatory Cognitive Embodied Systems

Ana Almeida e PaivaInstituto Superior Tecnico

Rino FalconeISTC-CNR

Georg DorffnerOFAI

Christian BalkeniusLund University

Joachim HoffmannUniversity of Wuerzburg

Fabio AdezioNoze s.r.l.

Juergen SchmidhuberScuola Universitaria Professionale

della Svizzera Italiana

Boicho KokinovNew Bulgarian University

www.mindraces.org


GOALInvestigate different anticipatory cognitive mechanisms and architectures

PRACTICAL VALIDATIONSimulations

Robotic platforms

TARGET DOMAINEmbodied systems

FOCUSAnticipation


CORE IDEAAnticipation is a key component of cognition

CORE IDEAComparative analysis of models of anticipation, and development of models

CRITICAL ISSUES‘Anticipatory emotions’

Performance metrics

CRITICAL ISSUESExpectation-driven attention, monitoring, and controlGoal-directed behaviour, pro-active planning, and prediction by analogy


Mind RACESfrom Reactive to Anticipatory Cognitive Embodied Systems

Comparative Analysis & Development of Anticipatory Systems

Think: Anticipation

www.mindraces.org


GNOSYSAn Abstraction Architecture for Cognitive Agents

John G. TaylorKing’s College London

Hanspeter MallotUniversity of Tuebingen

Christos EmmanouilidisZenon S.A.

Pietro MorassoDIST, University of Genova

Panos TrahaniasFoundation for Research and Technology - Hellas

www.ics.forth.gr/gnosys


GOALComplete conceptual architecture for cognitive agents

PRACTICAL VALIDATIONPatrolling, incident detection,

and tracking in unknown outdoor environments

TARGET DOMAINRobot navigation & surveillance

FOCUSComplete integration of the perception-knowledge acquisition-abstraction-reasoning- action generationcycle


CORE IDEAAbstraction creates concept hierarchiesReasoning then makes inferences for virtual actions

CORE IDEAAction selection based on greatest reward

Goals-oriented model to fuse user-derived and agent-derived tasks/goals

CRITICAL ISSUESInspired by

human cognitive processes

CRITICAL ISSUESAttention (complexity, priority, novelty, goals)Reinforcement learningActions modify current knowledge through perception


GNOSYSAn Abstraction Architecture for Cognitive Agents

Conceptual Architecture

Think: Abstraction, Concepts, Reasoning

www.ics.forth.gr/gnosys


Based on limited information

CoSy JAST RobotCub COSPAL MACS SPARK Mind GNOSYSRACES

Architecture

RoboticExploration

Robotic Manipulation

IncrementalLearning

Attention

Affordances

Language/Gesture

Multi-Agent


Based on limited information

CoSy JAST RobotCub COSPAL MACS SPARK Mind GNOSYSRACES

Self-ModifyingArchitecture

Self-Organization

SymbolicRepresentations

Neural Networks

DynamicalSystems


Project THINK …

CoSy

JAST

RobotCub

COSPAL

MACS

SPARK

Mind RACES

GNOSYS

Integration

Shared Understanding

Open Cognitive Humanoid System

Action-dependent Perception

Affordances

Emergent Spatio-temporal Patterns

Anticipation

Abstraction, Concepts, Reasoning

cognitive systems in the fp6 ist programme ist 2004, the ...holk cruse university of bielefeld...

Documents