Motor Theory Remnants

April 3, 2012

Dirty Work• Project Reports #5 to turn in.

• On Thursday, we’ll talk about the muscles that control articulation…

• And do a slightly messy static palatography demo

• At the end of today, we’ll do the USRI evaluations.

Another Piece of the Puzzle• Another interesting finding which has been used to argue for the “speech is special” theory is duplex perception.

• Take an isolated F3 transition:

and present it to one ear…

Do the Edges First!• While presenting this spectral frame to the other ear:

Two Birds with One Spectrogram

• The resulting combo is perceived in duplex fashion:

• One ear hears the F3 “chirp”;

• The other ear hears the combined stimulus as “da”.

Duplex Interpretation• Check out the spectrograms in Praat.

• Mann and Liberman (1983) found:

• Discrimination of the F3 chirps is gradient when they’re in isolation…

• but categorical when combined with the spectral frame.

• (Compare with the F3 discrimination experiment with Japanese and American listeners)

• Interpretation: the “special” speech processor puts the two pieces of the spectrogram together.

fMRI data• Benson et al. (2001)

• Non-Speech stimuli = notes, chords, and chord progressions on a piano

fMRI data• Benson et al. (2001)

• Difference in activation for natural speech stimuli versus activation for sinewave speech stimuli

Mirror Neurons• In the 1990s, researchers in Italy discovered what they called mirror neurons in the brains of macaques.

• Macaques had been trained to make grasping motions with their hands.

• Researchers recorded the activity of single neurons while the monkeys were making these motions.

• Serendipity:

• the same neurons fired when the monkeys saw the researchers making grasping motions.

• a neurological link between perception and action.

• Motor theory claim: same links exist in the human brain, for the perception of speech gestures

Moving On…• One important lesson to take from the motor theory perspective is:

• The dynamics of speech are generally more important to perception than static acoustic cues.

• Note: visual chimerism and March Madness.

Auditory Chimeras• Speech waveform + music spectrum:

• Music waveform + speech spectrum:

frequency bands

1 2 4 8 16 32

frequency bands

1 2 4 8 16 32

Source: http://research.meei.harvard.edu/chimera/chimera_demos.html

Originals:

Auditory Chimeras• Speech1 waveform + speech2 spectrum:

• Speech2 waveform + speech1 spectrum:

frequency bands

1 2 4 6 8 16

frequency bands

1 2 4 6 8 16

Originals:

Motor Theory, in a nutshell• The big idea:

• We perceive speech as abstract “gestures”, not sounds.

• Evidence:

1. The perceptual interpretation of speech differs radically from the acoustic organization of speech sounds

2. Speech perception is multi-modal

3. Direct (visual, tactile) information about gestures can influence/override indirect (acoustic) speech cues

4. Limited top-down access to the primary, acoustic elements of speech

Audition (or, how we hear things)

April 3, 2012

How Do We Hear?• The ear is the organ of hearing. It converts sound waves into electrical signals in the brain.

• the process of “audition”

• The ear has three parts:

• The Outer Ear

• sound is represented acoustically (in the air)

• The Middle Ear

• sound is represented mechanically (in solid bone)

• The Inner Ear

• sound is represented in a liquid

The Ear

Outer Ear Fun Facts• The pinna, or auricle, is a bit more receptive to sounds from the front than sounds from the back.

• It functions primarily as “an earring holder”.

• Sound travels down the ear canal, or auditory meatus.

• Length 2 - 2.5 cm

• Sounds between 3500-4000 Hz resonate in the ear canal

• The tragus protects the opening to the ear canal.

• Optionally provides loudness protection.

• The outer ear dead ends at the eardrum, or tympanic membrane.

The Middle Ear

eardrum

the hammer (malleus)

the anvil (incus)

the stirrup (stapes)

The Middle Ear• The bones of the middle ear are known as the ossicles.

• They function primarily as an amplifier.

• = increase sound pressure by about 20-25 dB

• Works by focusing sound vibrations into a smaller area

• area of eardrum = .55 cm2

• area of footplate of stapes = .032 cm2

• Think of a thumbtack...

Concentration• Pressure (on any given area) = Force / Area

• Pushing on a cylinder provides no gain in force at the other end...

• Areas are equal on both sides.

• Pushing on a thumb tack provides a gain in force equal to A1 / A2.

• For the middle ear , force gain

• .55 / .032 17

Leverage• The middle ear also exerts a lever action on the inner ear.

• Think of a crowbar...

• Force difference is proportional to ratio of handle length to end length.

• For the middle ear:

• malleus length / stapes length

• ratio 1.3

Conversions• Total amplification of middle ear 17 * 1.3 22

• increases sound pressure by 20 - 25 dB

• Note: people who have lost their middle ear bones can still hear...

• With a 20-25 dB loss in sensitivity.

• (Fluid in inner ear absorbs 99.9% of acoustic energy)

• For loud sounds (> 85-90 dB), a reflex kicks in to attenuate the vibrations of the middle ear.

• this helps prevent damage to the inner ear.

The Attenuation Reflex• Requires 50-100 msec of reaction time.

• Poorly attenuates sudden loud noises

• Muscles fatigue after 15 minutes or so

• Also triggered by speaking

tensor tympani

stapedius

The Inner Ear• In the inner ear there is a snail-shaped structure called the cochlea.

• The cochlea:

• is filled with fluid

• consists of several different membranes

• terminates in membranes called the oval window and the round window.

Cochlea Cross-Section

• The inside of the cochlea is divided into three sections.

• In the middle of them all is the basilar membrane.

Contact

• On top of the basilar membrane are rows of hair cells.

• We have about 3,500 “inner” hair cells...

• and 15,000-20,000 “outer” hair cells.

How does it work?• On top of each hair cell is a set of about 100 tiny hairs (stereocilia).

• Upward motion of the basilar membrane pushes these hairs into the tectorial membrane.

• The deflection of the hairs opens up channels in the hair cells.

• ...allowing the electrically charged endolymph to flow into them.

• This sends a neurochemical signal to the brain.

An Auditory Fourier Analysis• Individual hair cells in the cochlea respond best to particular frequencies.

• General limits:

20 Hz - 20,000 Hz

• Cells at the base respond to high frequencies;

• Cells at the apex respond to low.tonotopic organization of the

cochlea

How does this work?• Hermann von Helmholtz (again!) first proposed the place theory of cochlear organization.

• Original idea: one hair cell for each frequency.

• a.k.a. the “resonance theory”

• But...we can perceive more frequencies than we have hair cells for.

• The rate theory emerged as an alternative:

• Frequency of cell firing encodes frequencies in the acoustic signal.

• a.k.a. the “frequency theory”

• Problem: cell firing rate is limited to 1000 Hz...

Synthesis• The volley theory attempted to salvage the frequency rate proposal.

• Idea: frequency rates higher than 1000 Hz are “volleyed” back and forth between individual hair cells.

• There is evidently considerable evidence for this proposal.

Traveling Waves (in the ear!)• Last but not least, there is the traveling wave theory.

• Idea: waves of different frequencies travel to a different extent along the cochlea.

• Like wavelength:

• Higher frequency waves are shorter

• Lower frequency waves are longer

The Traveling Upshot• Lower frequency waves travel the length of the cochlea...

• but higher frequencies cut off after a short distance.

• All cells respond to lower frequencies (to some extent),

• but fewer cells respond to high frequency waves.

• Individual hair cells thus function like low-pass filters.

Hair Cell Bandwidth

• Each hair cell responds to a range of frequencies, centered around an optimal characteristic frequency.

Frequency Perception• In reality, there is (unfortunately?) more than one truth--

• Place-encoding (traveling wave theory) is probably more important for frequencies above 1000 Hz;

• Rate-encoding (volley theory) is probably more important for frequencies below 1000 Hz.

• Interestingly, perception of frequencies above 1000 Hz is much less precise than perception of frequencies below 1000 Hz.

• Match this tone:

• To the tone that is twice the frequency:

Higher Up• Now try it with this tone:

• Compared to these tones:

• Idea: listeners interpret pitch differences as (absolute) distances between hair cells in the cochlea.

• Perceived pitch is expressed in units called mels.

• Twice the number of mels = twice as high of a perceived pitch.

• Mels = 1127.01048 * ln (1 + F/700)

• where acoustic frequency (F) is expressed in Hertz.

The Mel Scale

Equal Loudness Curves• Perceived loudness also depends on frequency.