symmetrical eeg–fmri imaging by sparse...
TRANSCRIPT
Symmetrical EEG–fMRI imaging by sparse regularization
Thomas OberlinChristian Barillot, Rémi Gribonval, Pierre Maurel
HEMISFER ProjectPANAMA and VISAGES teams
Inria Rennes
EUSIPCO’15 – Nice, France
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 1 / 21
Introduction
Introduction : EEG–fMRI
EEG� Measure of the synchronous electrical activity of groups of neurons (pyramidal cells)� Instantaneous measures, perfect temporal resolution but poor spatial information� Easy-to-use, cheap technique
fMRI (through BOLD signal)� BOLD (Blood-Oxygen Level Dependent) measures the hemodynamic activity� High spatial resolution� More involved and expensive technique (non-invasive, though)
Aims :� Coupling both modalities to achieve a high spatio-temporal resolution� Perspectives : used it for Neurofeedback and rehabilitation (Hemisfer project)
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 2 / 21
Introduction
EEG–fMRI coupling : quick review
Acquisition� Sequential : no artefacts, reproducibility issues� Simultaneous : need a strong pre-processing step a
a. Laufs et al., "Recent advances in recording electrophysiological data simultaneously with mag-netic resonance imaging", Neuroimage, 2008
Types of coupling� Asymmetrical (most of the studies) : one modality is used as a prior to inform theother. Examples : fMRI-constrained EEG a, EEG-augmented fMRI b.
� Symmetrical (some recent attempts) : uses equally both modalities. Tools : bayesianfusion c, Kalman-like filtering d. Main difficulty : complexity of the neurovascularcoupling
a. Liu et al., "fMRI–EEG integrated cortical source imaging by use of time-variant spatial con-straints", Neuroimage, 2008
b. De Munck et al., "Interactions between different EEG frequency bands and their effect onalpha–fMRI correlations", Neuroimage, 2009
c. Daunizeau et al., "Symmetrical event-related EEG/fMRI information fusion in a variationalBayesian framework", Neuroimage, 2007
d. Deneux et al., "EEG-fMRI fusion of paradigm-free activity using Kalman filtering", Neuralcomputation, 2010
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 3 / 21
Introduction
Aims and objectives
Solving the inverse problem� Simultaneous and symmetrical coupling of EEG and fMRI� Formulation : sparsity-penalized regression already used in EEG/MEG a b c
� Optimization through proximal alorithms d
a. Gaudes et al., "Structured sparse deconvolution for paradigm free mapping of functional MRIdata", ISBI’2012
b. Ou et al., "A distributed spatio-temporal EEG/MEG inverse solver", Neuroimage, 2009c. Gramfort et al., "Mixed-norm estimates for the M/EEG inverse problem using accelerated
gradient methods", IPMI’2011d. Combettes et al., "Proximal splitting methods in signal processing", Springer, 2011
Next step : (dictionnary) learning, application to Neurofeedback� Using this high resolution imaging modality for learning models, dictionnaries,patterns, etc
� Apply it to EEG–only neurofeedback with a pre-training with EEG–fMRI.Applications : brain rehabilitation for depression, strokes, etc
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 4 / 21
Outline
Outline
1 Introduction
2 The neurovascular couplingDistributed source model in EEGModeling the BOLD signal in fMRISolving the inverse problem
3 OptimizationForward-backward algorithmChoice of penalty
4 Numerical resultsModel and simulationsContribution of the couplingChoice of parametersChoice of the penalizationRobustness to BOLD false positives/negatives
5 Conclusion
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 5 / 21
The neurovascular coupling
Outline
1 Introduction
2 The neurovascular couplingDistributed source model in EEGModeling the BOLD signal in fMRISolving the inverse problem
3 Optimization
4 Numerical results
5 Conclusion
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 6 / 21
The neurovascular coupling Distributed source model in EEG
Distributed source model in EEG
Neuronal activity X ≡ S dipoles(sources) distributed on the corticalsurface, in the normal direction
Measures E : a linear mixing of the sources
E = GX + NE , (1)
where� X ∈ RS×T is the activity of the sources (unknown)
� E ∈ RN×T is the electrical activity measured on the scalp (known)� G is the leadfield mixing matrix N × S (known)
� The noise term NE ∈ RN×T is Gaussian (with estimated covariance matrix)
S ≈ 10000 sources, N ≈ 100 electrodes, T ≈ 1000 time instants
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 7 / 21
The neurovascular coupling Modeling the BOLD signal in fMRI
Modeling the BOLD signal in fMRI
� BOLD signal F : linked to the neuronal activity X by a cascade of highly complexphysiological processes� Modeling of the coupling : "balloon model" 1
� The linearization of this model 2 leads to a convolutive model
Yi(t) = (Xi ? h)(t), (2)
where h is the standard hemodynamical response function (HRF)� Global formulation (for all voxels) :
F = Q(XH) + NF ,
whereF ∈ RS×U (U ≈ 20 number of time instants)Hij = h(ti − uj ) is the linear operator associated to the convolution in timeQ is an interpolation operatorNF is the noise term, which is still supposed to be Gaussian, with known covariance
1. Friston et al., "Nonlinear responses in fMRI : the Balloon model, Volterra kernels, and otherhemodynamics", Neuroimage, 2000
2. Robinson et al., "Bold responses to stimuli : dependence on frequency, stimulus form, ampli-tude, and repetition rate", Neuroimage, 2006
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 8 / 21
The neurovascular coupling Solving the inverse problem
Solving the inverse problem
ModelingWe suppose that the BOLD signal is measured at the same position of the sources, whichgives : {
E = GX + NEF = XH + NF .
(3)
Solving the inverse problemMinimizing a data-fit term and a regularization term :
X∗ = argmin(α
2 ‖E − GX‖2F +1− α2 ‖F − XH‖2F + λφ(X)
), (4)
� α tunes the tradeoff between each modality� λ controls the regularization term
Numerical solution through proximal iterative algorithms.
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 9 / 21
Optimization
Plan
1 Introduction
2 The neurovascular coupling
3 OptimizationForward-backward algorithmChoice of penalty
4 Numerical results
5 Conclusion
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 10 / 21
Optimization Forward-backward algorithm
Implementing the optimization
Proximal gradient algorithm (aka "forward-backward") a b
a. Daubechies et al., "An iterative thresholding algorithm for linear inverse problems with asparsity constraint", Com. Pure App. Math., 2004
b. Combettes et al., "Signal recovery by proximal forward-backward splitting", MMS, 2005
� Minimize f (x) + φ(x) with f diff., L-Lipschitz gradient, φ proper and convex� Here f is quadratic with L ≤ α ‖G∗G‖+ (1− α) ‖HH∗‖ (‖.‖spectral norm)
� At iteration k, replacing f (X) by a quadratic majorant ∇f (Xk)T (X − Xk) + L2 ‖X − Xk‖2F :
Xk+1 = argmin12
∥∥∥X −(
Xk −1L∇f (Xk)
)∥∥∥2F+λ
Lφ(X). (5)
� A unique solution in terms of proximal operator proxφ(X) = arg minY∈Rn 12 ‖X − Y‖2F + φ(Y ) :
Xk+1 = prox λL φ
(Xk −1L∇f (Xk)), (6)
� Special kind of Majorization/Minimization algorithms� Asymptotic linear convergence� The only difficult step : compute the proximal operator.� At each step, only a few matrix/matrix multiplications
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 11 / 21
Optimization Choice of penalty
Choice of penalty
`1-norm (Lasso) φ(X) = ‖X‖1 =∑
i,j |Xij | : soft thresholding (ST)
[proxµ‖.‖1(x)]j = sign(xj)(|xj | − µ)+ =
(1− µ
|xj |
)+
xj .
`12 norm (group Lasso) a ‖X‖12 =∑
i ‖Xi‖2 =∑
i
√∑j X 2
ij : group ST
a. A. Gramfort et al., "Mixed-norm estimates for the M/EEG inverse problem using acceleratedgradient methods", Phys. in med. and biol., 2012
[proxµ‖.‖12(X)]i =
(1− µ
‖Xi‖2
)+
Xi .
Other types of penalties� Total variation (2D on the cortex)� Sparsity in a dictionnary : φ(X) = ‖DX‖1
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 12 / 21
Optimization Choice of penalty
Generalized shrinkages, nonconvex penalties
Approximating the `0 quasi-norm φ(x) = #{xi 6= 0}...
Empirical Wiener shrinkage a, log penalty b...
−10 −8 −6 −4 −2 0 2 4 6 8 100
2
4
6
8
10
l1
log a=0.1
log a=0.3
Wiener
log a=1
−3 −2 −1 0 1 2 3
−2.5
−2
−1.5
−1
−0.5
0
0.5
1
1.5
2
Identity
Soft
log a=0.1
log a=0.3
Wiener
log a=1
a. Siedenburg et al., "Audio declipping with social sparsity", ICASSP’14b. Selesnick et al., "Sparse signal estimation by maximally sparse convex optimization", IEEE
TSP, 2014
... or the `02 quasi-norm φ(X) = #{‖Xi‖ 6= 0}Replacing group soft-thresholding by structured empirical Wiener shrinkage (SEW)
[Sµ(X)]i =
(1−
(µ
‖Xi‖2
)2)
+
Xi . (7)
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 13 / 21
Numerical results
Plan
1 Introduction
2 The neurovascular coupling
3 Optimization
4 Numerical resultsModel and simulationsContribution of the couplingChoice of parametersChoice of the penalizationRobustness to BOLD false positives/negatives
5 Conclusion
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 14 / 21
Numerical results Model and simulations
Model and simulations
� 3-layer spherical head model, 272 dipoles distributed on the cortex� Simulation of 3 punctual active sources (2 damped waves, 1 LF)� EEG : sampling frequency 500 Hz, 31 electrodes� fMRI : sampling frequency 1 Hz� White Gaussian noise : SNR of 2 dB for EEG and -18 dB for fMRI� Evaluation metrics : SNR and Spatial Error (SE, cm)
0 0.1 0.2 0.3 0.4−0.01
0
0.01
0.02
0.03
time (s)
Am
plit
ude (
µ V
)
True sources X
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 15 / 21
Numerical results Model and simulations
Model and simulations
� 3-layer spherical head model, 272 dipoles distributed on the cortex� Simulation of 3 punctual active sources (2 damped waves, 1 LF)� EEG : sampling frequency 500 Hz, 31 electrodes� fMRI : sampling frequency 1 Hz� White Gaussian noise : SNR of 2 dB for EEG and -18 dB for fMRI� Evaluation metrics : SNR and Spatial Error (SE, cm)
electrodes
dipoles
active sources
3-layer spherical head model
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 15 / 21
Numerical results Model and simulations
Model and simulations
� 3-layer spherical head model, 272 dipoles distributed on the cortex� Simulation of 3 punctual active sources (2 damped waves, 1 LF)� EEG : sampling frequency 500 Hz, 31 electrodes� fMRI : sampling frequency 1 Hz� White Gaussian noise : SNR of 2 dB for EEG and -18 dB for fMRI� Evaluation metrics : SNR and Spatial Error (SE, cm)
0 0.1 0.2 0.3 0.4−0.1
0
0.1
0.2
time (s)
Am
plit
ud
e (
µ V
)
0 5 10 15 20
0
5
10
x 10−3
time (s)B
OL
D c
ha
ng
e (
a.u
.)
EEG measures (noise-free) fMRI measures (noise-free)
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 15 / 21
Numerical results Model and simulations
Model and simulations
� 3-layer spherical head model, 272 dipoles distributed on the cortex� Simulation of 3 punctual active sources (2 damped waves, 1 LF)� EEG : sampling frequency 500 Hz, 31 electrodes� fMRI : sampling frequency 1 Hz� White Gaussian noise : SNR of 2 dB for EEG and -18 dB for fMRI� Evaluation metrics : SNR and Spatial Error (SE, cm)
0 0.1 0.2 0.3 0.4−0.1
0
0.1
0.2
time (s)
Am
plit
ud
e (
µ V
)
0 5 10 15 20
−0.01
0
0.01
time (s)B
OL
D c
ha
ng
e (
a.u
.)
EEG measures (noisy) fMRI measures (noisy)
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 15 / 21
Numerical results Contribution of the coupling
Benefit of the coupling
0 0.1 0.2 0.3 0.4 0.5−0.01
−0.005
0
0.005
0.01
0.015
0.02
0.025
time (s)
Am
plitu
de (
µ V
)
EEG only, SNR = 6, SE = 6.3 cm
0 0.1 0.2 0.3 0.4 0.50
0.002
0.004
0.006
0.008
0.01
time (s)
Am
plitu
de (
µ V
)
fMRI only, SNR = 0 dB, SE = 0 cm
0 0.1 0.2 0.3 0.4 0.5−0.005
0
0.005
0.01
0.015
0.02
0.025
time (s)
Am
plitu
de (µ
V)
EEG–fMRI, SNR = 17 dB, SE = 0 cm
0 0.1 0.2 0.3 0.4 0.5−0.01
−0.005
0
0.005
0.01
0.015
0.02
0.025
0.03
time (s)
Am
plitu
de (µ
V)
Reconstruction with known sources (least-squares),SNR = 26 dB, SE = 0 cm
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 16 / 21
Numerical results Choice of parameters
Choice of parameters
� Heuristics for choosing parameters :
α∗ =Uσ2
FTσ2
E + Uσ2F,
λ < λM :=α2 ‖E‖
2F + 1−α
2 ‖F‖2F
φ(X∗) ,
� Numerical experiments
λ (log10)
α (
log
10
)
−5 −4 −3 −2 −1
−5
−4
−3
−2
−1
λ (log10)
α (
log
10
)
−5 −4 −3 −2 −1
−5
−4
−3
−2
−1
λ (log10)
α (
log
10
)
−5 −4 −3 −2 −1
−5
−4
−3
−2
−1
σE = 0.001σF = 0.001
σE = 0.001σF = 0.005
σE = 0.01σF = 0.001
� Conclusion :Large range of suitable parametersHeuristic not very precise, but satisfactoryMust be completed by cross-validation
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 17 / 21
Numerical results Choice of the penalization
Influence of the penalty φ
`1SNR-out = 16 dB
SE = 0 cm
`12SNR-out = 8 dB
SE = 5 cm
SEWSNR-out = 24 dB
SE = 0 cm
λ (log10)
-4 -3 -2 -1 0
α (
log10)
-4
-3
-2
-1
`12
λ (log10)
-4 -3 -2 -1 0
α (
log10)
-4
-3
-2
-1
log, a = 1λ (log10)
-4 -3 -2 -1 0
α (
log10)
-4
-3
-2
-1
SEW
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 18 / 21
Numerical results Robustness to BOLD false positives/negatives
Robustness to BOLD false positives/negatives
noise-free BOLD signal asymmetrical reconstruction symmetrical reconstruction
SNR = 25 dB, SE = 0 cm SNR = 24 dB, SE = 0 cm
SNR = 4 dB, SE = 6.4 cm SNR = 18 dB, SE = 0 cm
SNR = 3 dB, SE = 3 cm SNR = 19 dB, SE = 0 cm
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 19 / 21
Conclusion
Conclusion
On the coupling� A (too ?) simple model, linear in time and space� Adapting the HRF to indivuals and brain locations
A simple linear and structured inverse problem� Importance of the penalty, the parameters� Nice behavior of non-convex relaxations (but more sensitive to the choice ofparameters)
Perspectives� Investigation with real data (need for nonlinear neurovascular coupling)� Image fusion in remote sensing a
� Nonconvex structured penalties in EEG b
a. Q. Wei et al., “Hyperspectral and multispectral image fusion based on a sparse representation”,IEEE Trans. on Geoscience and Remote Sensing, 2015
b. F. Costa et al., “EEG Source Localization Based on a Structured Sparsity Prior and a PartiallyCollapsed Gibbs Sampler”, submitted, 2015
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 20 / 21
Conclusion
Thanks for your attention
Any question ?
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 21 / 21
Conclusion
Thanks for your attention
Any question ?
T. Oberlin (Univ. Toulouse) EEG–fMRI imaging EUSIPCO’15 – Nice, France 21 / 21