Cochlear ImplantsCochlear ImplantsSTATS 19 SEM 2. 263057202. STATS 19 SEM 2. 263057202.

Talk 8.Talk 8.

A.L. Yuille.A.L. Yuille.

Dept. Statistics. Dept. Statistics. yuille@stat.ucla.eduyuille@stat.ucla.edu

Nerve Cells and the Brain:Nerve Cells and the Brain:

• The brain is made up of two types of cells.The brain is made up of two types of cells.

• Neurons and Glial Cells.Neurons and Glial Cells.

• Neurons are the basic elements of the Neurons are the basic elements of the nervous system. Information travels along nervous system. Information travels along them by electric impluses.them by electric impluses.

• Neurons transmit to other neurons by Neurons transmit to other neurons by synapses.synapses.

• Glial cells play a supporting role. Maintain Glial cells play a supporting role. Maintain brain structure. Do not conduct electricity.brain structure. Do not conduct electricity.

Nerve Cells and the Brain.Nerve Cells and the Brain.

• Neurons and Neurons and Networks of Networks of Neurons.Neurons.

• Brain has Brain has 100,000,000,000 100,000,000,000 neurons.neurons.

• Each communicates Each communicates with several 1,000 with several 1,000 other neurons.other neurons.

Nerve Cells and the BrainNerve Cells and the Brain

• Number of nerve cells Number of nerve cells 100,000,000,000 is about the 100,000,000,000 is about the number of trees in the Amazon number of trees in the Amazon rainforest.rainforest.

• Number of nerve cell Number of nerve cell connections connections 1,000,000,000,000,000 is 1,000,000,000,000,000 is about the number of leaves in about the number of leaves in rainforest.rainforest.

• Number of connections in the Number of connections in the world’s telephone system world’s telephone system (biggest machine on the (biggest machine on the planet).planet).

• Brain is the most complicated Brain is the most complicated system in the Universe.system in the Universe.

Magnetic Resonance Imaging Magnetic Resonance Imaging (MRI)(MRI)• MRI can get 3D pictures of MRI can get 3D pictures of

organs in the brain – by organs in the brain – by placing patient in a huge placing patient in a huge magnet.magnet.

• Hydrogen atoms (2/3 Hydrogen atoms (2/3 body) absorb and give off body) absorb and give off magnetic energy.magnetic energy.

• Calculate images from Calculate images from variations in how this variations in how this magnetic energy is magnetic energy is absorbed and emitted in absorbed and emitted in the body.the body.

• Low energy, non-invasive.Low energy, non-invasive.

Functional Magentic Functional Magentic Resonance Imaging (fMRI).Resonance Imaging (fMRI).• Use fMRI to measure Use fMRI to measure

blood flow in the brain.blood flow in the brain.• Active areas require Active areas require

more blood. fMRI can more blood. fMRI can measure the oxygen measure the oxygen content of the blood.content of the blood.

• Molecules in the blood Molecules in the blood cells respond differently cells respond differently to the magnetic field to the magnetic field depending on how depending on how much oxygen they are much oxygen they are carrying.carrying.

Language and the Brain.Language and the Brain.

• Moghul Emperor Akbar Khan (17Moghul Emperor Akbar Khan (17thth century).century).

• Put 12 babies with dumb nurses in a Put 12 babies with dumb nurses in a castle.castle.

• At age 12, brought them to his At age 12, brought them to his palace – and found they did not palace – and found they did not speak any language at all.speak any language at all.

Language and Grammar Language and Grammar

• Linguistics: rules of syntax (grammar) Linguistics: rules of syntax (grammar) operate independent of meaning operate independent of meaning (semantics).(semantics).

• Chomsky’s claim: people have an Chomsky’s claim: people have an innate, universal, system of syntax innate, universal, system of syntax which makes laws governing which makes laws governing elementary sentences.elementary sentences.

• ““Colourful green ideas sleep furiously” Colourful green ideas sleep furiously” – grammatical, but not meaningful.– grammatical, but not meaningful.

Language and the BrainLanguage and the Brain

• Left-hemisphere dominant for language. Left-hemisphere dominant for language. (strongest for right-handed males).(strongest for right-handed males).

• Broca’s area. Identified in 1861. Damage to Broca’s area. Identified in 1861. Damage to area impairs speech production.area impairs speech production.

• Speech production: but also involves some Speech production: but also involves some speech comprehension. (Syntactic?)speech comprehension. (Syntactic?)

• Wernicke’s area. Identified 1873. Damage Wernicke’s area. Identified 1873. Damage to it affects speech comprehension.to it affects speech comprehension.

• But also production. (Semantic?)But also production. (Semantic?)

Cochlear Implants.Cochlear Implants.

• The Most Successful Implant Device. The Most Successful Implant Device. • Provides good artificial input to the Provides good artificial input to the

auditory system, but does not solve auditory system, but does not solve the Cortical Part of Language. the Cortical Part of Language.

• A pre-linguistically deafened child or A pre-linguistically deafened child or adult find it very hard to use them.adult find it very hard to use them.

• First six years are crucial to language First six years are crucial to language acquisition and usage.acquisition and usage.

Cochlear ImplantsCochlear Implants

• Can probably train completely deaf Can probably train completely deaf humans to hear, provided it is done humans to hear, provided it is done early.early.

• Experiments on kittens than are born Experiments on kittens than are born deaf.deaf.

• Implants given at age 3-4 months.Implants given at age 3-4 months.• Several months of training, kittens Several months of training, kittens

behaviour showed they “hear” sounds behaviour showed they “hear” sounds normally. (Brain activity confirms this).normally. (Brain activity confirms this).

Cochlear ImplantsCochlear Implants

• 500,000 Americans have severe hearing loss.500,000 Americans have severe hearing loss.• 32,000 people world wide would implants. 32,000 people world wide would implants.

Cost $30-50,000.Cost $30-50,000.• Most common hearing loss (85%) is caused by Most common hearing loss (85%) is caused by

damage to cochlear hair cells in the inner ear.damage to cochlear hair cells in the inner ear.• Damage cn be genetic, or caused by disease Damage cn be genetic, or caused by disease

(measles, meningitis, injury, aging, drugs).(measles, meningitis, injury, aging, drugs).• For adults, implants can be miraculous. 10 For adults, implants can be miraculous. 10

seconds like pinball. In two minutes, could seconds like pinball. In two minutes, could hear and have conversations.hear and have conversations.

Cochlear Implants.Cochlear Implants.

• Tiny microphone attached near the Tiny microphone attached near the ear. Implants send sounds through a ear. Implants send sounds through a processor, back to a transmitter, that processor, back to a transmitter, that delivers the sound through electrodes, delivers the sound through electrodes, stimulating the auditory nerve.stimulating the auditory nerve.

• 40,000 Americans are deaf-blind. 40,000 Americans are deaf-blind. They seem to benefit from Cochlear They seem to benefit from Cochlear implants.implants.

Cochlear.Cochlear.

• Recipients of Cochlear implants suggestRecipients of Cochlear implants suggesta true hearing percept (i.e. not like having to a true hearing percept (i.e. not like having to

guess what the sound means).guess what the sound means).• Awareness of sound in the environment.Awareness of sound in the environment.• Can combine with lip reading, or be used Can combine with lip reading, or be used

independently.independently.• Typically understand 60% speech within Typically understand 60% speech within

two weeks, 80% within a few months two weeks, 80% within a few months (sufficient for telephone use).(sufficient for telephone use).

Cochlear Implant.Cochlear Implant.

• Takes over the operation of the Takes over the operation of the Cochlear – not just a hearing-aid Cochlear – not just a hearing-aid (amplifier).(amplifier).

• Damaged hair cells can cause Damaged hair cells can cause damage of adjacent auditory neurons damage of adjacent auditory neurons (very bad).(very bad).

• But usually auditory neurons remain.But usually auditory neurons remain.

Auditory System.Auditory System.

• Cochlear nerves from ears transmit Cochlear nerves from ears transmit acoustic signals to cochlear nucleus in acoustic signals to cochlear nucleus in lower brainstem.lower brainstem.

• Then proceed to temporal lobes of the Then proceed to temporal lobes of the cerebral cortex.cerebral cortex.

Cochlear and the EarCochlear and the Ear

• Outer Ear – Middle Ear – Inner Ear.Outer Ear – Middle Ear – Inner Ear.• Outer Ear picks up acoustic pressure Outer Ear picks up acoustic pressure

waves.waves.• Converts them into mechanical Converts them into mechanical

vibrations by a series of small bones in vibrations by a series of small bones in middle ear.middle ear.

• In Inner Ear, the Cochlear (snail In Inner Ear, the Cochlear (snail shaped cavity full of fluid) transforms shaped cavity full of fluid) transforms them into fluid vibrations.them into fluid vibrations.

Cochlear and the Ear Cochlear and the Ear

• Pressure variations within the fluid of the Pressure variations within the fluid of the Cochlear lead to displacement of the basilar Cochlear lead to displacement of the basilar membrane.membrane.

• The displacements contain information about The displacements contain information about the frequency of the acoustic signal.the frequency of the acoustic signal.

• There are hair cells attached to the basilar There are hair cells attached to the basilar membrane.membrane.

• Bending of hair cells releases electrochemical Bending of hair cells releases electrochemical substance which causes auditory neurons to substance which causes auditory neurons to fire.fire.

• Signals presence of excitation at a particular Signals presence of excitation at a particular sitesite

Auditory FequenciesAuditory Fequencies

• Sound can be decomposed in Sound can be decomposed in frequency components.frequency components.

• Different frequency components cause Different frequency components cause maximum vibration at different points maximum vibration at different points along the basilar membrane.along the basilar membrane.

• Low frequencies cause biggest Low frequencies cause biggest amplitudes at the apex of basilar amplitudes at the apex of basilar membrane (fig 3).membrane (fig 3).

• High frequencies near the base High frequencies near the base (stapes).(stapes).

Audio.Audio.

Audio.Audio.

Auditory Frequences.Auditory Frequences.

• Cochlear implant must activate a range of Cochlear implant must activate a range of frequencies.frequencies.

• Signal processor in implant must Signal processor in implant must decompose the auditory signal into decompose the auditory signal into frequencies (Fourier Analysis).frequencies (Fourier Analysis).

• Activate places in the Cochlear by Activate places in the Cochlear by electrodes to correspond to input electrodes to correspond to input frequencies.frequencies.

• Loudness depends on how many auditory Loudness depends on how many auditory neurons are activated. Increase strength neurons are activated. Increase strength of electric current.of electric current.

Cochlear ImplantsCochlear Implants

• Electrode arrays inserted in the scala Electrode arrays inserted in the scala tympani to depths of 22-30 mm.tympani to depths of 22-30 mm.

• Often 22 electrodes are used Often 22 electrodes are used (frequency channels).(frequency channels).

• First devices used a single electrode, First devices used a single electrode, was not ideal. was not ideal.

Cochlear ImplantsCochlear Implants

• Transmission to implant.Transmission to implant.

• Transcutaneous. Radio frequency Transcutaneous. Radio frequency link. Transmitter and implanted link. Transmitter and implanted receiver held in place by a magnet.receiver held in place by a magnet.

• Percutaneous. Transmits to Percutaneous. Transmits to electrodes by plug connections.electrodes by plug connections.

PerformancePerformance

• Before – sentence recognition 30% or Before – sentence recognition 30% or less.less.

• After – aim between 70% to 100%. After – aim between 70% to 100%. Patients can often fill in missing Patients can often fill in missing words by context.words by context.

Children Children

• Implant will help child:Implant will help child:• (1) Produce speech (feedback).(1) Produce speech (feedback).• (2) Understand speech.(2) Understand speech.

• Implants give steady improvement for Implants give steady improvement for both skills. (Up to four years).both skills. (Up to four years).

• The earlier the better.The earlier the better.• Fastest improvement for postlingually Fastest improvement for postlingually

deafened children.deafened children.

Factors affecting Factors affecting performanceperformance

• Duration of Deafness – the shorter Duration of Deafness – the shorter the better.the better.

• Age of onset of deafness – better if Age of onset of deafness – better if postlingual.postlingual.

• Age at implantation – Best to implant Age at implantation – Best to implant prelingual children early (2 years prelingual children early (2 years min).min).

Cochlear History.Cochlear History.

• Pythagoras – sound is an oscillation of the Pythagoras – sound is an oscillation of the air. (6air. (6thth century BC). century BC).

• Fallopio (1561) discover the Cochlear.Fallopio (1561) discover the Cochlear.• Cochlear Figure.Cochlear Figure.• Corti (1851) found hair cells on the organ Corti (1851) found hair cells on the organ

of Corti.of Corti.• Helmholtz suggested that the organ of Helmholtz suggested that the organ of

Corti vibrates in response to different Corti vibrates in response to different sound frequencies.sound frequencies.

Cochlear HistoryCochlear History

• Von Bekesy (Nobel Prize 1961) showed Von Bekesy (Nobel Prize 1961) showed that vibrations of the Cochlear did that vibrations of the Cochlear did correspond to sound waves.correspond to sound waves.

• Different parts of the Organ of Corti Different parts of the Organ of Corti were sensitive to different tones.were sensitive to different tones.

• Zwicker (1957) showed that the Zwicker (1957) showed that the auditory system organized sounds into auditory system organized sounds into 24 channels. 30,000 nerve fibers.24 channels. 30,000 nerve fibers.

Hearing LossHearing Loss

• Two types of hearing loss.Two types of hearing loss.

• (I) Conductive Hearing Loss: Damage (I) Conductive Hearing Loss: Damage to the apparatus that transmits to the apparatus that transmits sound to the Cochlea (e.g. eardrum). sound to the Cochlea (e.g. eardrum). Can be treated by hearing aids or Can be treated by hearing aids or surgery to raise volume.surgery to raise volume.

• (II) Sorineural: Destruction of the hair (II) Sorineural: Destruction of the hair cells within the organ of Corti.cells within the organ of Corti.

Hearing LossHearing Loss

• How many electrodes needed? 30,000 for all How many electrodes needed? 30,000 for all the nerve fibers? 24 for Zwicker’s channels? the nerve fibers? 24 for Zwicker’s channels? 6 based on frequency analysis?6 based on frequency analysis?

• Kiang (1965) discovered that an entire Kiang (1965) discovered that an entire population of nerve fibers must be used to population of nerve fibers must be used to transmit a single frequency of sound (single transmit a single frequency of sound (single fibers can produce impluses at most 300 fibers can produce impluses at most 300 times per second, but speech involves times per second, but speech involves frequencies up to 4,000 cycles per second).frequencies up to 4,000 cycles per second).

Newer DiscoveriesNewer Discoveries

• New discoveries keep suggesting ways to New discoveries keep suggesting ways to improve Cochlear Implants.improve Cochlear Implants.

• Better understanding of how the nerve Better understanding of how the nerve fibers encode the speech frequencies.fibers encode the speech frequencies.

• Understanding the temporal pattern of the Understanding the temporal pattern of the nerve fibers.nerve fibers.

• Detailed understanding of how the hair Detailed understanding of how the hair cells change sound into electrical nerve cells change sound into electrical nerve impulses (2002).impulses (2002).

Speech versus VisionSpeech versus Vision

• Speech input is simpler than Vision input.Speech input is simpler than Vision input.• Acoustic signals can be decomposed in Acoustic signals can be decomposed in

terms of frequencies. 30,000 hair cells.terms of frequencies. 30,000 hair cells.• Images cannot be described so simply Images cannot be described so simply

(jpeg encoding). 10,000,000 retinal cells.(jpeg encoding). 10,000,000 retinal cells.• In both cases, fixing the input doesn’t In both cases, fixing the input doesn’t

help if the language/vision area of the help if the language/vision area of the cortex have not developed.cortex have not developed.

Speech versus Vision Speech versus Vision

• But for speech, prelingual deaf children But for speech, prelingual deaf children can learn to speak and hear.can learn to speak and hear.

• Cochlear implants provide sufficient Cochlear implants provide sufficient information to train the language areas of information to train the language areas of the cortex?the cortex?

• And/or Chomsky may be right, syntax may And/or Chomsky may be right, syntax may be largely innate.be largely innate.

• Retinal implants have a very long way to Retinal implants have a very long way to go to catch up to Cochlear implants. go to catch up to Cochlear implants.