the effect of conditional probability of chord progression in western music corpus on brain...
TRANSCRIPT
THE EFFECT OF CONDITIONAL PROBABILITY OF CHORD PROGRESSION IN WESTERN MUSIC
CORPUS ON BRAIN RESPONSE: AN MEG STUDY
2010-09-11Department of Brain and Cognitive Sciences | SNU
Seung-Goo KIM
THE EFFECT OF CONDITIONAL PROBABILITY OF CHORD PROGRESSION IN WESTERN MUSIC
CORPUS ON BRAIN RESPONSE: AN MEG STUDY
2010-09-11Department of Brain and Cognitive Sciences | SNU
Seung-Goo KIM
NEURONAL CURRENT
http://www.cnsforum.com/content/pictures/imagebank/hirespng/placement_EEG_leads.png
NEURONAL CURRENT
http://www.cnsforum.com/content/pictures/imagebank/hirespng/placement_EEG_leads.png
NEURONAL CURRENT
http://www.cnsforum.com/content/pictures/imagebank/hirespng/placement_EEG_leads.png
http://www.nmr.mgh.harvard.edu/meg/talks.php
NEURONAL CURRENT
http://www.cnsforum.com/content/pictures/imagebank/hirespng/placement_EEG_leads.png
INSTRUMENTS
http://www.calvin.edu/academic/psych/experiential/research.html
http://www.virtualmedicalcentre.com/healthinvestigations.asp?sid=11
http://www.nmr.mgh.harvard.edu/meg/talks.php
rMusical scale§ Tonal systems within an octave such
as major or minor
rChord§ Simultaneous tones such as (C,E,G)
rChord function§ The function of a chord within a scale
• (C,E,G) is Tonic (I) in C Major and also Dominant (V) in F Major.
rChord progression§ Changes of chord functions§ “Syntactic” rules govern this progression in Western music
CHORD PROGRESSION
rMusical scale§ Tonal systems within an octave such
as major or minor
rChord§ Simultaneous tones such as (C,E,G)
rChord function§ The function of a chord within a scale
• (C,E,G) is Tonic (I) in C Major and also Dominant (V) in F Major.
rChord progression§ Changes of chord functions§ “Syntactic” rules govern this progression in Western music
CHORD PROGRESSION
PROCESS OF MUSICAL SYNTAX
rKoelsch, 2000, J. Cog. Neurosci.§ Regular (50%), N6 at the 3rd pos (25%), N6 at the 5th pos (25%)
C: I I6 IV V7 I C: I I6 N6 V7 I C: I I6 IV V7 N6
Koelsch et al., 2000, J. Cog. Neurosci.
PROCESS OF MUSICAL SYNTAX
rKoelsch, 2000, J. Cog. Neurosci.§ Regular (50%), N6 at the 3rd pos (25%), N6 at the 5th pos (25%)
C: I I6 IV V7 I C: I I6 N6 V7 I C: I I6 IV V7 N6
Koelsch et al., 2000, J. Cog. Neurosci.
NEURAL GENERATORS OF ERAN
rLocated usingMEG & fMRI
rSeveral studieslocated IFG
rSometimesbilaterallythus simply EAN(esp. in women)
Maess et al, 2001, NN; Koelsch et al., 2005, TICS; Koelsch et al., 2003, Neuroreport.
NEURAL GENERATORS OF ERAN
rLocated usingMEG & fMRI
rSeveral studieslocated IFG
rSometimesbilaterallythus simply EAN(esp. in women)
Maess et al, 2001, NN; Koelsch et al., 2005, TICS; Koelsch et al., 2003, Neuroreport.
AUTOMATICITY & ATTENTION
rAttended condition§ detect the intensity,
not the deviant chord§ (Task-‐irrelevant
-‐target condition: to avoid P300b)
rUnattended condition§ ignoring all chord
with ‘hard’ reading comprehension task
Loui et al., 2005, Cog. Brain Res.
MUSICAL TRAINING
Koelsch et al., 2002, Psychophysiol.
rERAN was greater in experts§ Not in deviant instrument MMN
MUSICAL TRAINING
Koelsch et al., 2002, Psychophysiol.
rERAN was greater in experts§ Not in deviant instrument MMN
SHORT-TERM-EXPERIENCE: HABITUATION
r2 Hours of chords with silent movie» Koelsch, Jentschke, 2008, Brain Research.
§ Target(supertonic)-‐irrelevant task(timber detection)
rDeclined; but not abolished
Koelsch et al., 2008, Brain res.
OTHER IN/OUT-OF-KEY CHORDS
rNeapolitan 6th is out-‐of-‐key chord§ What if with other chords?
Koelsch et al., 2007, Psychophysiol.
OTHER IN/OUT-OF-KEY CHORDS
rNeapolitan 6th is out-‐of-‐key chord§ What if with other chords?
Koelsch et al., 2007, Psychophysiol.
EAN VS. MMN
rMismatch Negativity (MMN)§ Well-‐known ERP response to the rare stimuli (ex: S, S, S, D, S, S, ….)
• Similar in terms of latency, related neural generators,
§ The rarity depends to the short-‐term rarity• Same stimuli could be either a standard or a deviant
Opitz et al., 2002, Neuroimage.
EAN VS. MMN
rEAN: Invariant to the short-‐term rarity§ The irregular chords § The rarity of the progression is rather dependent§ MMN can be elicited by roving paradigm
• EAN can be evoked even with equiprobability
Garrido et al., 2008, Neuroimage.
Roving Paradigm:Firstly, it’s oddballLastly, it’s standard
NEWLY LEARNED PITCH PATTERNS
rUsing artificial scales (Bohlen-‐Pierce)§ Standard pitch pattern (70%) & Deviant pitch pattern (20%)§ 100 sequences X 10 runs = total 40 min
Loui et al., 2009, J. Neurosci.
NEWLY LEARNED PITCH PATTERNS
rUsing artificial scales (Bohlen-‐Pierce)§ Standard pitch pattern (70%) & Deviant pitch pattern (20%)§ 100 sequences X 10 runs = total 40 min
Loui et al., 2009, J. Neurosci.
NEWLY LEARNED PITCH PATTERNS
rUsing artificial scales (Bohlen-‐Pierce)§ Standard pitch pattern (70%) & Deviant pitch pattern (20%)§ 100 sequences X 10 runs = total 40 min
Loui et al., 2009, J. Neurosci.
NEWLY LEARNED PITCH PATTERNS
rUsing artificial scales (Bohlen-‐Pierce)§ Standard pitch pattern (70%) & Deviant pitch pattern (20%)§ 100 sequences X 10 runs = total 40 min
Loui et al., 2009, J. Neurosci.
NEWLY LEARNED PITCH PATTERNS
rUsing artificial scales (Bohlen-‐Pierce)§ Standard pitch pattern (70%) & Deviant pitch pattern (20%)§ 100 sequences X 10 runs = total 40 min
Loui et al., 2009, J. Neurosci.
NEWLY LEARNED PITCH PATTERNS
rUsing artificial scales (Bohlen-‐Pierce)§ Standard pitch pattern (70%) & Deviant pitch pattern (20%)§ 100 sequences X 10 runs = total 40 min
Loui et al., 2009, J. Neurosci.
MOTIVATIONS
rPossible underlying mechanism of EAN has been suggested as ‘Probabilistic learning’ depending on the previous exposures.§ Unlike the MMN depending on the present situation
» Koelsch, 2009, Psychophy.
§ Learning of chord pattern could evoke EAN» Loui et al., 2009, J. Neurosci.
rHowever, the direct relation of the frequency of exposure in terms of chord progression and the physiological responses has not been investigated yet.
AIMS OF THE PRESENT STUDY
rIf the EANm responses are dependent on the probability in previous exposure
rIf this effect, if any, interacts with the musical expertise and the individual ability of discriminating chord progressions
APPROXIMATION OF FREQUENCYrConditional Probability from J. S. Bach’s Chorale
§ Empirical probability as an approximation of frequencies in exposure of music listeners in Western classical context
P(I|V)=0.75; P(vi|V)=0.11; P(ii|V)=0.05Rohrmeier, 2007, Sound and Music Computing Conference, Greece.
TonicP(I|V)=0.75
SubmediantP(vi|V)=0.11
SupertonicP(ii|V)=0.05
I vi ii V
MATERIAL
• 1 chord=0.6 sec (100 BPM)•Modulated to 12 keys• Consecutively presented
• Note that (vi) and (ii) occurred before the last chord to control on-‐line rarity of chord itself
TonicP(I|V)=0.75
SubmediantP(vi|V)=0.11
SupertonicP(ii|V)=0.05
I vi ii V
MATERIAL
• 1 chord=0.6 sec (100 BPM)•Modulated to 12 keys• Consecutively presented
• Note that (vi) and (ii) occurred before the last chord to control on-‐line rarity of chord itself
TonicP(I|V)=0.75
SubmediantP(vi|V)=0.11
SupertonicP(ii|V)=0.05
I vi ii V
MATERIAL
• 1 chord=0.6 sec (100 BPM)•Modulated to 12 keys• Consecutively presented
• Note that (vi) and (ii) occurred before the last chord to control on-‐line rarity of chord itself
TonicP(I|V)=0.75
SubmediantP(vi|V)=0.11
SupertonicP(ii|V)=0.05
I vi ii V
MATERIAL
• 1 chord=0.6 sec (100 BPM)•Modulated to 12 keys• Consecutively presented
• Note that (vi) and (ii) occurred before the last chord to control on-‐line rarity of chord itself
PARTICIPANTS
r11 Non-‐musicians & 9 Musicians (N=20)§ after exclusion for excessive MEG or EOG artifacts§ As no male musicians could be recruited, to control the sex ratio
between groups, all participants were female§ Other 2 participants were discarded for exceed EOG rejections
Mean (SD) N Age (Yr) Handedness (EI)
Musical training (Yr)
Practice (Hour/Week)
Non-‐musicians 11 24.93 (2.5) 91.49 (9.27) 3.63 (1.63) 1.0 (3.0)
Musicians 9 23.50 (3.4) 88.64 (12.78) 18.67 (4.0) 10.22 (6.74)
MEG RECORDING
r1 Block = 100 trails (6 min)§ 3 kinds of sequences x 30 = 90 trials§ 3 kinds of “staccato” sequences = 10 trials
rTask: click the left mouse button on “staccato” chord (randomly at the 2nd ~5th position)§ To preserve the arousal level of the participants
rTotal 6 Blocks with breaks (App. 1 Hour)§ 180 trails were presented for each condition
rElekta Neuromag 306-‐channel whole-‐head MEG§ Anatomical landmarks, HPI coils and points upon the head digitized§ Sampling rate=600Hz; BPF=0.1~200 Hz
MEG RECORDING
r1 Block = 100 trails (6 min)§ 3 kinds of sequences x 30 = 90 trials§ 3 kinds of “staccato” sequences = 10 trials
rTask: click the left mouse button on “staccato” chord (randomly at the 2nd ~5th position)§ To preserve the arousal level of the participants
rTotal 6 Blocks with breaks (App. 1 Hour)§ 180 trails were presented for each condition
rElekta Neuromag 306-‐channel whole-‐head MEG§ Anatomical landmarks, HPI coils and points upon the head digitized§ Sampling rate=600Hz; BPF=0.1~200 Hz
BEHAVIORAL TEST
rIdentify the last chord (Tonic/ Submediant/ Supertonic) of the sequences using a keypad in the same environment with MEG recording
r1 Block= 3 kinds x 12 keys = 36 questionsrTotal 3 blocks (App. 15 min)
rExamples for each chord progressions were presented before each block
SIGNAL PROCESSING
rSpatio-‐temporal filter applied to eliminate environmental noise (MaxFilterTM)
rAll blocks transformed to the head position of the first block
rEpoch: -‐200 ~ +500 msec to the target onset§ Epochs with EOG artifacts discarded (<10%)
rAveraged across blocks for each condition§ Mean averaged trials for each condition >150
rBand pass filter§ 1~20 Hz with zero phase for ECD analysis
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESA
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESA
All in-‐key chords
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESA
All in-‐key chords
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESA
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESA
Supertonic
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESA
Supertonic
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESA
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESATonic
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESATonic
Submediant
rMultiple ECDs were fitted as followings:§ Generators of P2m activities were fitted using genetic algorithm
implemented in BESA® (Brain Electrical Source Analysis; NeuroScan)• For the latency of 180-‐190 ms in the temporal regions
§ Additional two bilateral ECDs for EANm responses were fitted• The time-‐window of 140-‐220 ms after onset• Seeding from the frontal regions bilaterally for the condition of
Supertonic as the frontal activities were shown consistently in the previous literatures (accumulated goodness of fitting was over 80%)
§ The activities of ECDs were estimated for each condition
MULTIPLE ECD ANALYSIS W/ BESATonic
SubmediantSupertonic
STATISTICAL ANALYSES
rRepeated measure ANOVA§ Within-‐subject: Chord (3 levels); Between-‐subject: Group (2 levels)
rGeneral Linear Model (GLM)§ Additional quadratic model to test the effects of training
ECD LOCATIONS
Mean (SEM) in head coordinatex (mm) y (mm) z (mm)
lHG -‐42.43 (1.16) -‐0.52 (1.57) 46.61 (1.25)
rHG 46.06 (1.30) 6.35 (1.90) 47.69 (1.29)
lIFG -‐36.27 (2.15) 18.13 (4.97) 65.77 (3.52)
rIFG 41.99 (2.42) 19.89 (4.89) 63.56 (3.54)
Hotelling’s t2 : t2(3, 36)= 39.07, p< 0.0001); Right: t2 (3, 36)= 21.62, p< 0.001)
ECD ACTIVITIES (N=20)
Shading indicates the time-‐window of [140, 220] ms after onsetActivities didn’t differ across hemispheres (t(19)= -‐0.63, p> 0.5) thus pooled
EFFECTS OF CHORD & TRAINING
rEANm response was defined as the mean of absolute values in the time window [0.14, 0.22]
rRepeated measure ANOVA on the EANm responses rThe effect of chord functions
§ F(1.188, 21.389)= 34.636, p< 0.0001 (Greenhouse-‐Geisser’s corr.)
rThe effect of musical training with an interaction § F(1.188, 21.389)= 4.304, p= 0.0044 (Greenhouse-‐Geisser’s corr.)
CORRELATION WITH PROB. & TRAINING
The quadratic model considering the effects of musical training and interaction explained significantly better (F(2, 56)= 6.4390, p= 0.0030)
CORRELATION WITH BEHAVIORAL MEASURES
The correlation between the correct rates for Submediants and normalized EANm activities was significantly positive (r= 0.6244, p=0.0400)
EFFECT OF CONDITIONAL PROBABILITY
rEnding chord which has the lower conditional probability elicited the greater EANm response§ F(1.188, 21.389)= 34.636, p< 0.0001 (Greenhouse-‐Geisser’s corr.)
rIt is not due to acoustic deviance but syntactic§ All chord functions were played within the sequence before ending
rAlso the EANm responses correlates with the probability interacting with musical training§ Musicians: r= 0.4440; Non-‐musicians: r=0.3929
IFG AND PROBABILITY
sgKIM (2010) ThesisEAN/ MEG
Maess et al., (2001) NNERAN/ MEG
Opitz et al., (2002) NIMMN/ fMRI
Koelsch et al., (2005) NIERAN/ fMRI
RR R
INDIVIDUAL DIFFERENCES
rFor experts, the behavioral scores were not normally distributed § most of them reached the maxima (“Ceiling effect”)
rFor the normal group, Submediant scores highly correlate with the overall scores (r= 0.91)
rThe EANm could be considered as reflecting the individual competence of identifying subtle aberration§ Thus it could be suggested as the physiological index of the
probabilistic representation of individuals
LIMITATIONS
rmusicians showed enhanced responses to the short-‐term deviants & irregular chords (ERAN)
rThis might implicate the facilitated sensitivity to musical syntax or the familiarity to the corpus in the present study
Huron, 2006, Sweet AnticipationBaroque Corpora Popular music Corpora
LIMITATIONS
rEffects of the melody & chord function§ Different top voices might have confounded the effect of chord
functions§ But not likely,
rDifferent melody§ Doesn’t affect the
time-‐windowlater than 100 ms.
Koelsch & Jentschke (2010) J. Cog Neurosci.
CONCLUSION
rThe effects of the probability of chord progression on the brain responses were found§ The negative correlation between the conditional probability
and the corresponding neural response§ Also it was found to be more facilitated by the musical training
rThe current results suggest the physiological response as a reflection of the probabilistic representations on the musical syntax.
rMoreover, the results indicate that the probabilistic representation is related to the musical training as well as individual sensitivity.
CONCLUSION
rThe effects of the probability of chord progression on the brain responses were found§ The negative correlation between the conditional probability
and the corresponding neural response§ Also it was found to be more facilitated by the musical training
rThe current results suggest the physiological response as a reflection of the probabilistic representations on the musical syntax.
rMoreover, the results indicate that the probabilistic representation is related to the musical training as well as individual sensitivity.