electrophysiology as a brain measure of perceptual sensitivity and abstraction
TRANSCRIPT
Electrophysiology as a Brain Measure of Perceptual Sensitivity and Abstraction
/kæt/
what?
when?
where?
ElectrophysiologyHow it worksMapping toolDiscriminationCategorizationPredictionTiming
How electrophysiology works
Electrical Activity
Probing Electrical Activity
• Interfere with it
• Record it indirectly
• Record it directly
Zap!
Transcranial Magnetic StimulationK. L. Sakai et al. 2002, Neuron
Verb generationVerb generation Verb generation after 15 min practice
Verb generation after 15 min practice
Raichle & Posner, Images of Mind cover image
Bang!
Functional MagneticResonance Imaging (fMRI)
MRI studies brain anatomy.Functional MRI (fMRI) studies brain function.
MRI vs. fMRI
Source: Jody Culham, fMRI for Newbies
Electroencephalography (EEG)
Event-Related Potentials (ERPs)
s1 s2 s3
John is laughing.
Phillips, Kazanina, & Abada (2005) Cog. Br. Res.
The producers knew that the actress wished that …The producers knew which jokes the actress wished that ..
ERP Studies
• Cost ($$)– Relatively cheap equipment and maintenance
• Time Investment– Materials: e.g., 20 syllables; 128 x 4 target sentences; 256 fillers
– Acquisition: 2-3 hours x 12-24 participants
– Analysis: 1-2 hours/person preprocessing; rest automatizable
• Strengths– Unbeatable temporal resolution
– Easy to combine across participants
– Movement possible; longer studies more tolerable
– Interpretable results
• Disadvantages– Scalp topography ≠ localization
– … and limited sex appeal
Brain Magnetic Fields (MEG)
Brain magnetic fields recorded fully non-invasively by arrays of SQUID* detectors
[*Superconducting QUantum Interference Device]
V
skull
CSF
tissue
MEG
EEGB
- noninvasive measurement- direct measurement.
scalprecordingsurface
currentflow
orientationof magnetic field
Origin of the signal
How small is the signal?10-4
10-5
10-6
10-7
10-8
10-9
10-10
10-11
10-12
10-13
10-14
10-15
Earth field
Urban noise
Contamination at lung
Heart QRS
MuscleFetal heart
Spontaneous signal (-wave)
Signal from retina
Intrinsic noise of SQUID
Inte
nsity
of
mag
netic
si
gnal
(T)
Evoked signal
Biomagnetism
EYE (retina) Steady activity Evoked activity
LUNGS Magnetic contaminants
LIVER Iron stores
FETUS Cardiogram
LIMBS Steady ionic current
BRAIN (neurons) Spontaneous activity Evoked by sensory stimulation
SPINAL COLUMN (neurons) Evoked by sensory stimulation
HEART Cardiogram (muscle) Timing signals (His Purkinje system)
GI TRACK Stimulus response Magnetic contaminations
MUSCLE Under tension
160 SQUIDwhole-headarray
pickup coil & SQUIDassembly
Sensor layout: recording from 160 channelsResponse peak at 98ms after onset of an auditory stimulus, in the left and right temporal lobes.
Magnetic source imaging (MSI): MEG + MRIDipole fit at response peak, 98ms after onset of stimulus
(Halgren et al. 2002)
(Halgren et al. 2002)
MEG Studies
• Cost ($$)– Expensive equipment; little maintenance; liquid helium supply
• Time Investment– Materials: e.g., 20 syllables; 600 x 3 target sentences; 400 fillers
– Acquisition: 1-2 x 1-2.5 hours x 12 participants (+ structural MRI)
– Analysis: similar to EEG, more individual-specific analyses
• Strengths– Unbeatable temporal resolution; easy set-up
– Possibility of localization
• Disadvantages– Combining across individuals more difficult than in ERP studies
– Longer studies, movement limited
– Inverse problem for localization is very hard
– Sensitivity depends on orientation and depth of source
– Horrendously difficult with children, challenging w/ small heads
Electrophysiology in Syntax/Semantics
1. Classification Tool
2. Sensitive Timing Measure
Sue takes her coffee with cream and socks. N400 - semantic anomalyThe plane took we to paradise. P600 - syntactic anomaly
The hungry guests helped himself to food. P600
Prerequisite: response components sensitive to distinct information types
Mary praised Max’s of proof the theorem. ‘ELAN’, 100-200ms
No bills that the senators supported ever became law. controlThe bills that the senators supported ever became law. ~400ms responseThe bills that no senators supported ever became law. ~400ms response
Prerequisite: explicit process models of syntax
Electrophysiology as a Mapping Tool
Sensory Maps
Internal representations of the outside world. Cellular neuroscience has discovered a great deal in this area.
Sensory Maps
Internal representations of the outside world. Cellular neuroscience has discovered a great deal in this area.
Notions of sensory maps may be applicable to some aspects of human phonetic representations…
…but there has been little success in that regard, and we shouldn’t expect this to yield much.
Vowel Space
Obleser
Lahiri
J. Cogn. Neurosci.,16, 31-39 (2004)
M100• Elicited by any well-defined onset
• Varies with tone frequency
• Varies with F1 of vowels
• May vary non-linearly with VOT variation
• Functional value of time-code unclear
• No evidence of higher-level representations
100
110
120
130
140
150
0 1000 2000 3000 4000 5000
Frequency (Hz)
i (male) i (female) a (male) a (female) u (male) u (female)100
105
110
115
120
125
130
(Poeppel & Roberts 1996)
(Poeppel, Phillips et al. 1997)
(Phillips et al. 1995; Sharma & Dorman 1999)
Electrophysiological measures of discrimination
Mismatch Response
Latency: 150-250 msec.
Many-to-one ratio between standards and deviants
X X X X X Y X X X X Y X X X X X X Y X X X Y X X X...
Localization of Mismatch Response
(Phillips, Pellathy, Marantz et al., 2000)
[Radiological view - left is right]
Basic MMN elicitation
©Risto Näätänen
Mismatch Negativity (MMN)
Sams et al. 1985
Tiitinen et al. 1994
How does MMNlatency, amplitudevary with frequencydifference?
1000Hz tone std.
Different Dimensions of Sounds
• Length
• Amplitude
• Pitch
• …you name it …
Amplitude of mismatch response can be used as a measure of perceptual distance
Impetus for Language Studies
• If MMN amplitude is a measure of perceptual distance, then perhaps it can be informative in domains where acoustic and perceptual distance diverge…
Place of Articulation
• Acoustic variation: F2 & F3 transitions
Place of Articulation
• Acoustic variation: F2 & F3 transitions
[bæ] [dæ]
Place of Articulation
• Acoustic variation: F2 & F3 transitions
[bæ] [dæ]
withincategory
betweencategory
Place of Articulation
• Acoustic variation: F2 & F3 transitions
[bæ] [dæ]
withincategory
betweencategory
Categories in InfancyHigh Amplitude Sucking - 2 month olds
Eimas et al. 1971
20 vs. 40 ms. VOT - yes40 vs. 60 ms. VOT - no
Infants show contrast, but this doesn’t entail phonological knowledge
Place of Articulation
• No effect of category boundary on MMN amplitude (Sharma et al. 1993)
• Similar findings in Sams et al. (1991), Maiste et al. (1995)
but…
Näätänen et al. (1997)
e e/ö ö õ o
Relevance to Learning Models• Place of articulation continuum (Lalonde & Werker, 1988)
b -- d -- D
• 3 contrastsNative b -- dNon-native d -- DNon-phonetic b1 -- b5
• Conflicting results
– Dehaene-Lambertz 1997native contrast only
– Rivera-Gaxiola et al. 2000native + non-native contrasts
Dehaene-Lambertz Rivera-Gaxiola
Phillips (2001, Cognitive Science)
J. Cogn. Neurosci. 16:577-583 (2004)
Contrast andUnderspecification
Phonetic Category Effects
• Measures of uneven discrimination profiles
• Findings are mixed (…and techniques vary)
• Relies on assumption that effects of contrasts at multiple levels are additive,…plus the requirement that the additivity effect be strong enough to yield a statistical interaction
• Logic of next set of studies:
– Eliminate contribution of lower levels by isolating the many-to-one ratio at a more abstract level of representations
– Do this by introducing non-orthogonal variation among standards
Electrophysiological measures of abstraction
J. Cogn. Neuro., 12, 1038-1055 (2000)
/dæ/ /tæ/
Design
20ms 40ms 60ms
Fixed Design - Discrimination
Design
0ms 8ms 16ms 24ms 40ms 48ms 56ms 64ms
20ms 40ms 60ms
Fixed Design - Discrimination
Grouped Design - Categorization
Non-orthogonal within-category variation: precludes grouping via acoustic streaming.
Design
0ms 8ms 16ms 24ms 40ms 48ms 56ms 64ms
20ms 40ms 60ms
Fixed Design - Discrimination
Grouped Design - Categorization
20ms 28ms 36ms 44ms 60ms 68ms 76ms 84ms
Grouped Design - Acoustic Control
/dæ/ standard vs./dæ/ deviant
So what …
• Auditory cortex generator of MMF accesses representations that treat members of the same category as identical
• No indication of what might be the form of these representations, or where they might be stored
• MMF generator accesses multiple levels of representation
Sound Groupings
(Phillips, Pellathy, & Marantz, 2000)
pæ, tæ, tæ, kæ, dæ, pæ, kæ, tæ, pæ, kæ, bæ, tæ...
(Phillips, Pellathy, & Marantz, 2000)
More on features …
• Alternative account of the findings
– No feature-based grouping
– Independent MMF elicited by 3 low-frequency phonemes
/bæ/ /dæ/ /gæ/ /pæ/ /tæ/ /kæ/29% 29% 29% 4% 4% 4%
87.5% 12.5%
(Yeung & Phillips, 2004)
More on features …
• Next study distinguishes
– Phoneme-level frequency
– Feature-level status
/gæ//bæ/ /dæ/ /tæ/
37.5% 37.5% 12.5% 12.5%
(Yeung & Phillips, 2004)
More on features …
• Next study distinguishes
– Phoneme-level frequency
– Feature-level status
/gæ//bæ/ /dæ/ /tæ/
37.5% 37.5% 12.5% 12.5%
Phoneme-based classification
(Yeung & Phillips, 2004)
More on features …
• Next study distinguishes
– Phoneme-level frequency
– Feature-level status
/gæ//bæ/ /dæ/ /tæ/
37.5% 37.5% 12.5% 12.5%
Feature-based grouping
(Yeung & Phillips, 2004)
More on features …
• Design– N = 10
– Multiple exemplars, individually selected boundaries
– 2 versions recorded for all participants, reversing [±voice] value
– Acoustic control, with all VOT values in [-voice] range
/gæ//bæ/ /dæ/ /tæ/
37.5% 37.5% 12.5% 12.5%
Feature-based grouping
(Yeung & Phillips, 2004)
More on features …
Left-anteriorchannels
(Yeung & Phillips, 2004)
Kazanina et al., 2006
Proceedings of the NationalAcademy of Sciences, 103, 11381-6
Nina Kazanina, U. of Bristol
Russian vs. Korean
• Three series of stops in Korean:– plain (lenis) pa ta ka– glottalized (tense, long) p’a t’a k’a– aspirated ph tha kha
• Intervocalic Plain Stop Voicing:/papo/ [pabo] ‘fool’/ku papo/ [kbabo] ‘the fool’
• Plain stops:– Bimodal distribution of +VOT and –VOT tokens– Word-initially: always a positive VOT– Word-medially intervocalically: a voicing lead (negative VOT)
Identification/Rating
Discrimination
Black: p < .05White: n.s.
Russian vs. Korean
• MEG responses indicate that Russian speakers immediately map sounds from [d-t] continuum onto categories
• Korean speakers do not…… despite the fact that the sounds show bimodal distribution in their language
• Perceptual space reflects the functional status of sounds in encoding word meanings
Dupoux et al. (1999, Percep. Psychophys.)
J. Cogn. Neurosci. 12:635-647 (2000)
EXECTIVE SUITEEXECTIVE SUITE
French: CVC
egma egma
Japanese: *CVC
egma eguma
Early (~200ms)
Late (~600ms)
Dehaene-Lambertz et al. (2000, J. Cogn. Neurosci.)
Egma, egma, egma, egma, eguma, …
Mismatch Studies
• Perceptual experience of sound categories is mapped onto multiple levels of representation, at different degrees of abstraction
• Abstract category representations can be probed using non-orthogonal variation among sounds in an auditory mismatch paradigm more-or-less immediate access during speech perception
• Beware of hasty inferences to normal time course of processing
– Recurring standard sound establishes ‘model’ for parsing upcoming sounds
– Detecting deviance from model, even at abstract level, may be faster than normal analysis processes
– cf. related findings about ELAN (‘Early Left Anterior Negativity’) in syntactic ERP literature (Lau et al., 2006)
Electrophysiology and Prediction
McGurk Effect: auditory [pa] + visual [ka] = perceptual [ta]
Neurosci. Lett. 397:263-268 (2006)
[aba]
[ãba]
Electrophysiology and Timing
syntax > phonology
phonology > syntax
Science, 280: 572-574
Lateralized Readiness Potential (LRP)
Prospects
• Many possibilities for more sophisticated phonetic/phonological studies
– Mapping: what are the questions?
– Early transforms of acoustic space: already an active area
– Discrimination: needs a theory of variable access to levels of detail
– Categorization: can be used to investigate learning and abstraction
– Prediction: phonotactics is barely touched
– Timing: possible LRP studies of computing derived properties