hidden process models: decoding overlapping cognitive states with unknown timing
DESCRIPTION
Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing. Rebecca A. Hutchinson Tom M. Mitchell Carnegie Mellon University NIPS Workshops: New Directions on Decoding Mental States from fMRI Data December 8, 2006. Overview. Open questions we address: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/1.jpg)
Hidden Process Models:Decoding Overlapping Cognitive
States with Unknown Timing
Rebecca A. HutchinsonTom M. Mitchell
Carnegie Mellon University
NIPS Workshops: New Directions on Decoding Mental States from fMRI Data
December 8, 2006
![Page 2: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/2.jpg)
2
Overview
• Open questions we address:– Treating fMRI as the time series that it is.– Allowing the testing of hypotheses.
• Open questions we do NOT address:– Interpretability of time series or spatial representation
of activity.
• This talk– Motivation– HPMs (in 1 slide!)– Preliminary results
![Page 3: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/3.jpg)
3
Motivation
• Goal: connect fMRI to cognitive modeling.
• Cognitive Model:– Set of cognitive processes hypothesized to occur
during a given fMRI experiment.
• Cognitive Process:– Spatial-temporal hemodynamic response function.– Timing distribution relative to experiment landmarks
(like stimulus presentations and behavioral data).
![Page 4: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/4.jpg)
4
Study: Pictures and Sentences
• Task: Decide whether sentence describes picture correctly, indicate with button press.
• 13 normal subjects, 40 trials per subject.• Sentences and pictures describe 3 symbols: *,
+, and $, using ‘above’, ‘below’, ‘not above’, ‘not below’.
• Images are acquired every 0.5 seconds.
Read Sentence
View Picture Read Sentence
View PictureFixation
Press Button
4 sec. 8 sec.t=0
Rest
![Page 5: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/5.jpg)
5
One Cognitive Model
Read Sentence
View Picture Read Sentence
View PictureFixation
Press Button
4 sec. 8 sec.t=0
Rest
• ViewPicture – begins when picture stimulus is presented
• ReadSentence– begins when sentence stimulus is presented
• Decide– begins within 4 seconds of 2nd stimulus
ViewPicture or ReadSentence ViewPicture or ReadSentence
Decide
![Page 6: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/6.jpg)
6
Hidden Markov Models (HMMs)
Hidden Process Models (HPMs)
States (1 time point) Processes (time window)
Latent state sequence Latent process instances
No external input Use experiment design and behavioral data
State transition matrix Process-specific timing distributions
State-specific emission distributions
Process-specific response signatures
1 hidden Markov chain governs observed data
Process instances can overlap in space and time
Forward-backward training algorithm
EM training algorithm
![Page 7: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/7.jpg)
7
ViewPicture in Visual Cortex
![Page 8: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/8.jpg)
8
ReadSentence in Visual Cortex
![Page 9: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/9.jpg)
9
ViewPicture
![Page 10: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/10.jpg)
10
ReadSentence
![Page 11: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/11.jpg)
11
Decide
0 0.5 1 1.5 2 2.5 3 3.5
0 0 0 0 0.025 0.05 0.075 0.85
Seconds following the second stimulus
Multinomial probabilities on these time points
![Page 12: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/12.jpg)
12
Comparing ModelsHPM Avg. Test Set LL
PS -1.0784 * 10^6
PSD -1.0759 * 10^6
PS+S-D -1.0742 * 10^6
PSD+D- -1.0742 * 10^6
PSDB -1.0741 * 10^6
PSDyDn -1.0737 * 10^6
PSDyDnDc** -1.0717 * 10^6
PSDyDnDcB -1.0711 * 10^6
5-fold cross-validation, 1 subject
P = ViewPicture
S = ReadSentence
S+ = ReadAffirmativeSentence
S- = ReadNegatedSentence
D = Decide
D+ = DecideAfterAffirmative
D- = DecideAfterNegated
Dy = DecideYes
Dn = DecideNo
Dc = DecideConfusion
B = Button
** - This HPM can also classify Dy vs. Dn with 92.0% accuracy. GNBC gets 53.9%. (using the window from the second stimulus to the end of the trial)
![Page 13: Hidden Process Models: Decoding Overlapping Cognitive States with Unknown Timing](https://reader035.vdocument.in/reader035/viewer/2022062518/5681444c550346895db0e940/html5/thumbnails/13.jpg)
13
Conclusions
• Simultaneous estimation of spatial-temporal signature (HRF) and temporal onset of cognitive processes.
• Framework for principled comparison of different cognitive models in terms of real data.