alan l. yuille. ucla. dept. statistics and psychology. yuille neural prosthetic: mind reading. stats...

Alan L. Yuille.Alan L. Yuille.

UCLA. Dept. Statistics and UCLA. Dept. Statistics and Psychology.Psychology.

www.stat.ucla/~yuillewww.stat.ucla/~yuille

Neural Prosthetic: Mind Reading.Neural Prosthetic: Mind Reading.STATS 19 SEM 2. 263057202. STATS 19 SEM 2. 263057202. Talk 6.Talk 6.

Neural Prosthetic.Neural Prosthetic.

• Basic Idea:Basic Idea:

• Injury to spinal cord Injury to spinal cord prevents actions.prevents actions.

• But patient can still But patient can still think, or plan, the think, or plan, the actions.actions.

• If these plans can be If these plans can be decoded by a decoded by a microchip, then the microchip, then the patient can control a patient can control a robot limb, or other robot limb, or other device.device.

Mind Reading.Mind Reading.

• Basic Assumption: Mind = Brain.Basic Assumption: Mind = Brain.

• Brain consists of neurons transmitting Brain consists of neurons transmitting information by electrical activity.information by electrical activity.

• All thoughts have neuronal correlates. All thoughts have neuronal correlates.

• If you can decode the activity of neurons, If you can decode the activity of neurons, then you can read people’s thoughts.then you can read people’s thoughts.

Neural ActivityNeural Activity

• Decoding neural activity corresponds to Decoding neural activity corresponds to mind reading.mind reading.

• But how do neurons encode information?But how do neurons encode information?

• Two classic viewpoints:Two classic viewpoints:1.1. The grandmother cellThe grandmother cell that responds that responds

whenever you see your grandmother.whenever you see your grandmother.2.2. Activity encoded by populations of Activity encoded by populations of

neuronsneurons. Neural networks (holistic).. Neural networks (holistic).

Decoding Neurons.Decoding Neurons.

• It seems clear that some brain activity It seems clear that some brain activity is localized in specific areas – e.g. is localized in specific areas – e.g. face recognition.face recognition.

• Also some neurons respond to specific Also some neurons respond to specific stimuli – e.g. to faces but not to dogs.stimuli – e.g. to faces but not to dogs.

• There might even be a Clinton cell…There might even be a Clinton cell…

• There is a lot of plasticity.There is a lot of plasticity.

Motion PlanningMotion Planning

• Motion Planning : experiments (1990) Motion Planning : experiments (1990) showed that populations of neurons in showed that populations of neurons in the motor cortex predicted the direction the motor cortex predicted the direction of movement.of movement.

• Better ways to interpret neuronal Better ways to interpret neuronal activity – get monkeys to perform activity – get monkeys to perform simple tasks, measure neuronal activity, simple tasks, measure neuronal activity, perform Bayesian inference. (Brown perform Bayesian inference. (Brown University).University).

Plasticity and Feedback Plasticity and Feedback HelpsHelps• The decoding is used to drive a The decoding is used to drive a

device.device.• The monkey gets feedback by The monkey gets feedback by

observing that the device is not doing observing that the device is not doing what it wanted.what it wanted.

• The monkey can somehow correct its The monkey can somehow correct its thought patterns, so that the decoding thought patterns, so that the decoding works. works.

• Like learning to ride a bicycle.Like learning to ride a bicycle.

Movement PlanningMovement Planning

• These movement plans, or thoughts, These movement plans, or thoughts, need not be conscious.need not be conscious.

• Novel actions usually require conscious Novel actions usually require conscious thought.thought.

• But most common actions do not.But most common actions do not.

• Neuronal correlates of consciousness.Neuronal correlates of consciousness.

Neural Prosthetics.Neural Prosthetics.

• Only a few Universities have serious neural Only a few Universities have serious neural prosthetics programs.prosthetics programs.

• This talk describes work by Richard This talk describes work by Richard Andersen’s Laboratory at Caltech. Figures Andersen’s Laboratory at Caltech. Figures copied from the lab’s webpage.copied from the lab’s webpage.

• Michael Black at Brown University. Michael Black at Brown University. (Decoding).(Decoding).

Goal of Neural Prosthetic Goal of Neural Prosthetic System.System.

• Read the Read the plans plans from the from the cortex.cortex.

• Activate Activate an an external external device.device.

Implanting Electrodes.Implanting Electrodes.

• Measurement is done by Measurement is done by inserting electrodes into inserting electrodes into the cortex. These the cortex. These transmit to an external transmit to an external device.device.

• Will the implanted Will the implanted electrodes stay in the electrodes stay in the right place? Will they right place? Will they cease functioning over cease functioning over time?time?

• But the electrodes only But the electrodes only need to measure activity need to measure activity (passive). They do not (passive). They do not need to stimulate need to stimulate neurons (epilepsy).neurons (epilepsy).

How to Implant Electrodes?How to Implant Electrodes?• Neurosurgery:Neurosurgery:

• Deep implants of Deep implants of electrodes to electrodes to simulate simulate (treatment for (treatment for Parkinson’s Parkinson’s disease). disease).

• No pain receptors No pain receptors in the brain.in the brain.

Miniaturized Chip.Miniaturized Chip.

• Chip must measure Chip must measure neuronal activity.neuronal activity.

• Action potentials.Action potentials.

Where the Output Goes.Where the Output Goes.

• Chip implanted Chip implanted inside the inside the brain.brain.

• Wires go out Wires go out and activate and activate external external devices.devices.

Posterior Parietal Cortex.Posterior Parietal Cortex.

• Earliest place where Earliest place where the thought to make a the thought to make a movement is mademovement is made..

• Size of little fingernail.Size of little fingernail.• About 50,000,000 About 50,000,000

neurons.neurons.

• Others work with motor Others work with motor cortex. But this can cortex. But this can atrophy with disuse.atrophy with disuse.

Plans made in PPC. Sent to Motor Cortex. Transmitted to Spinal Cord.

Cognitive Plans Cognitive Plans

• Planning Planning and Visual and Visual CoordinatesCoordinates..

• Advantage Advantage of PPT.of PPT.

Parietal Reach Region.Parietal Reach Region.

• Reach, Reach, Move Move Eyes, Eyes, Grasp.Grasp.

Spatial Representation Spatial Representation Known.Known.• Knowing spatial representations makes Knowing spatial representations makes

it easier to read the neuronal code.it easier to read the neuronal code.

Neural activity as Monkey Neural activity as Monkey Thinks.Thinks.

• Monkey Monkey is trained is trained to to perform perform tasks.tasks.

• Waiting Waiting period period when when monkey monkey plans.plans.

Close the Loop by Giving Close the Loop by Giving Feedback.Feedback.

• Monkey has Monkey has feedback.feedback.

• Monkey can Monkey can adjust adjust neural neural activity to activity to improve improve performanceperformance..

Improve Recordings.Improve Recordings.

• Moveable Moveable ProbeProbe

• Need to adjustNeed to adjust

Electrodes to Electrodes to take take

tissue tissue movement movement

into account.into account.

Computer Probe.Computer Probe.

• Adjust Probes.Adjust Probes.

• 1,000 electrodes.1,000 electrodes.

Automatically Automatically optimize positions.optimize positions.

• Microfluidic drug Microfluidic drug delivery. To delivery. To remove scar remove scar tissue.tissue.

Decoding Neural Activity.Decoding Neural Activity.

• Local Field Local Field Potentials.Potentials.

Local Field Potentials.Local Field Potentials.

• Measure Field Measure Field Potentials.Potentials.

Tuning of Local Field Tuning of Local Field Potentials.Potentials.• Local Field Local Field

Potentials:Potentials:

Decoding LFP’s and SpikesDecoding LFP’s and Spikes

• Decoding.Decoding.

Informatics.Informatics.

• Real Time Decoding. Real Time Decoding.

• Analysis is similar to that for speech Analysis is similar to that for speech recognition.recognition.

• Information in space and time.Information in space and time.

• Similar techniques can be used.Similar techniques can be used.

Summary.Summary.

• Planned movements are represented Planned movements are represented in the Parietal Reach Region.in the Parietal Reach Region.

• Neural decoding enables these plans Neural decoding enables these plans to be read.to be read.

• Plasticity of neurons makes this easier, Plasticity of neurons makes this easier, training by feedback.training by feedback.

• Implanting device by deep brain Implanting device by deep brain surgery.surgery.

• Output wires control external device.Output wires control external device.