neural robot control

Project funded by the Future and Emerging Technologies arm of the IST ProgrammeProject funded by the Future and Emerging Technologies arm of the IST ProgrammeFET-Open schemeFET-Open scheme


Neural Robot ControlNeural Robot ControlNeural Robot ControlNeural Robot Control

Cornelius Weber

Hybrid Intelligent Systems

University of Sunderland

Talk at Nottingham Trent University, 8th December 2004

on the occasion of returning the MI competition trophy

Collaborators: Mark Elshaw, Alex Zochios, Chris Rowan and Stefan Wermter

Cornelius Weber

Hybrid Intelligent Systems

University of Sunderland

Talk at Nottingham Trent University, 8th December 2004

on the occasion of returning the MI competition trophy

Collaborators: Mark Elshaw, Alex Zochios, Chris Rowan and Stefan Wermter



ContentsContentsContentsContents

• Visual cortex & reinforcement network for docking

• Cortex self-imitation network for docking

• Imitation networks for multiple actions:

1-stage/2-stage hierarchical network

• Outlook

Example Task: DockingExample Task: DockingExample Task: DockingExample Task: Docking



Docking ArchitectureDocking ArchitectureInformation FlowInformation Flow

Docking ArchitectureDocking ArchitectureInformation FlowInformation Flow



Docking ArchitectureDocking ArchitectureTraining (1/3)Training (1/3)


unsupervised traininggenerative modelsparse distributed coding



V1 Receptive FieldsV1 Receptive Fields(training result)(training result)

V1 Receptive FieldsV1 Receptive Fields(training result)(training result)



winner

Comparison ofComparison ofResponse CharacteristicsResponse Characteristics

Comparison ofComparison ofResponse CharacteristicsResponse Characteristics

linear sparse competitive



Attractor Network:Attractor Network:Competition via RelaxationCompetition via Relaxation

Attractor Network:Attractor Network:Competition via RelaxationCompetition via Relaxation

weight profile activation profile

activation update

y(t+1) = f (Wlat y(t))





supervised training,attractor networkfor pattern completion



Docking ArchitectureDocking ArchitectureVisual SystemVisual System

Docking ArchitectureDocking ArchitectureVisual SystemVisual System





reinforcement trainingactor-critic model








• Outlook



Mirror NeuronMirror NeuronDocking ArchitectureDocking Architecture

Information FlowInformation Flow





unsupervised traininggenerative modeldistributed coding


TrainingTraining


TrainingTraining



supervised training,attractor networkfor prediction


TrainingTraining


TrainingTraining



Mirror Neuron Self-ImitationMirror Neuron Self-ImitationDocking ArchitectureDocking Architecture


Mirror Neuron Self-ImitationMirror Neuron Self-ImitationDocking ArchitectureDocking Architecture




Basal Ganglia vs. Motor CortexBasal Ganglia vs. Motor CortexBasal Ganglia vs. Motor CortexBasal Ganglia vs. Motor Cortex

Basal ganglia units are active during early task acquisition but not at a later stage (rat T maze decision task).

early: late:

Basal Ganglia ≙ state space?Motor cortex might take over BG function via self-imitation.

Jog et al. (1999)Jog et al. (1999) Science, 286, 1158-61 Science, 286, 1158-61

Jog et al. (1999)Jog et al. (1999) Science, 286, 1158-61 Science, 286, 1158-61

Docking via Mirror NeuronsDocking via Mirror NeuronsDocking via Mirror NeuronsDocking via Mirror Neurons








• Outlook



Simulated Robot EnvironmentSimulated Robot EnvironmentSimulated Robot EnvironmentSimulated Robot Environment



Imitation Model ChoiceImitation Model ChoiceImitation Model ChoiceImitation Model Choice



Areas of Motor- and Language Areas of Motor- and Language RepresentationsRepresentations

Areas of Motor- and Language Areas of Motor- and Language RepresentationsRepresentations

forward back left right

‘go’ ‘pick’ ‘lift’ all

individual unit’sreceptive fieldsin hidden area

motor units

language units



Areas of Task-Specific ActivationsAreas of Task-Specific ActivationsAreas of Task-Specific ActivationsAreas of Task-Specific Activations

‘go’ ‘pick’ ‘lift’


Recognition:

Production:

Activations agree with the Somatotopy-of-Action-Words Model.



Language InstructedLanguage InstructedImitative BehaviourImitative Behaviour

Language InstructedLanguage InstructedImitative BehaviourImitative Behaviour




Imitation Model ChoiceImitation Model ChoiceImitation Model ChoiceImitation Model Choice



Neuron’s Receptive Fields in HM AreaNeuron’s Receptive Fields in HM AreaNeuron’s Receptive Fields in HM AreaNeuron’s Receptive Fields in HM Area

motor units

4 SOM-area units

forward backward left right



Conclusion for Imitation NetworkConclusion for Imitation NetworkConclusion for Imitation NetworkConclusion for Imitation Network

A neural network as a generative model for sensory stimuli

• generates interactive action sequences

• allows for context dependent interactive action sequences








• Outlook



Outlook (1/2): Object-Background Outlook (1/2): Object-Background Separation for Enhanced Object Separation for Enhanced Object

LearningLearning

Outlook (1/2): Object-Background Outlook (1/2): Object-Background Separation for Enhanced Object Separation for Enhanced Object

LearningLearning



Outlook (2/2): Docking Range Outlook (2/2): Docking Range Extension by Neural Coordinate Extension by Neural Coordinate

TransformationsTransformations

Outlook (2/2): Docking Range Outlook (2/2): Docking Range Extension by Neural Coordinate Extension by Neural Coordinate

TransformationsTransformations

neural robot control

Documents

emerging technologies

imitation networks

early task acquisition

example task

multiple actions

basal ganglia state

f wlat ytproject

bg function