TECHNICAL NOTE

Experimental comparison of connectivity measureswith simulated EEG signals

Minna J. Silfverhuth • Heidi Hintsala •

Jukka Kortelainen • Tapio Seppanen

Received: 2 November 2011 / Accepted: 23 April 2012 / Published online: 22 May 2012

� International Federation for Medical and Biological Engineering 2012

Abstract Directional connectivity measures exist with

different theoretical backgrounds, i.e., information theoretic,

parametric-modeling based or phase related. In this paper, we

perform the first comparison in this extend of a set of con-

ventional and directed connectivity measures [cross-corre-

lation, coherence, phase slope index (PSI), directed transfer

function (DTF), partial-directed coherence (PDC) and

transfer entropy (TE)] with eight-node simulation data based

on real resting closed eye electroencephalogram (EEG)

source signal. The ability of the measures to differentiate the

direct causal connections from the non-causal connections

was evaluated with the simulated data. Also, the effects of

signal-to-noise ratio (SNR) and decimation were explored.

All the measures were able to distinguish the direct causal

interactions from the non-causal relations. PDC detected less

non-causal connections compared to the other measures.

Low SNR was tolerated better with DTF and PDC than with

the other measures. Decimation affected most the results of

TE, DTF and PDC. In conclusion, parametric-modeling-

based measures (DTF, PDC) had the highest sensitivity of

connections and tolerance to SNR in simulations based on

resting closed eye EEG. However, decimation of data has to

be carefully considered with these measures.

Keywords Biomedical signal processing �Computational biology � Electroencephalography

1 Introduction

Brain connectivity is a functional methodology to assess

connections between brain networks or areas. It is suitable

for detection and monitoring the subtle changes in brain

function. The early-stage signs of a serious neurological

illness leading to a disability may often emerge as a

functional abnormality, preceding detectable anatomical

damage in brain.

Conventionally used coherence and cross-correlation

measure strength of connection pair-wise between time

series measured from different locations, to confirm a

relationship (high value of coherence) in certain frequency

(band), between, e.g., EEG or fMRI time series (e.g., [11]).

However, by using these measures, direction cannot be

assigned. Therefore, there is no way to determine the flow

of information, to answer which brain area A is affecting

the processing of another brain area B.

Examples of directional connectivity measures [1, 8, 11,

12] are measures derived from information theory, like

transfer entropy (TE) [14], or parametric-modeling-based

Granger causality, directed transfer function (DTF) [6] and

partial-directed coherence (PDC) [2], or phase-related

measure phase slope index (PSI) [9]. TE has an advantage

of the model-free non-linear approach but requires com-

putational capacity. DTF and PDC are fast to compute but

model order estimation needs consideration [12]. PSI is fast

but a long data segment is necessary for reliable estimation.

Comparison of directional measures with different the-

oretical backgrounds (information theoretic, parametric-

modeling based or phase related) has only been performed

extensively in theory [11]. However, parametric-modeling-

based measures have been widely compared [e.g., 1, 4, 8,

12, 16], but in general with quite a simple connectiv-

ity patterns. In this paper, we explore the robustness of

M. J. Silfverhuth (&) � J. Kortelainen � T. Seppanen

Department of Computer Science and Engineering,

University of Oulu, Box 4500, 90014 Oulu, Finland

e-mail: minna.silfverhuth@oulu.fi

H. Hintsala

Centre for Environmental and Respiratory Health Research,

Institute of Biomedicine, University of Oulu,

BOX 5000, 90014 Oulu, Finland

123

Med Biol Eng Comput (2012) 50:683–688

DOI 10.1007/s11517-012-0911-y

cross-correlation, coherence, PSI, DTF, PDC and TE for

assessment of directional influences. Analysis is applied to

EEG-based multi-node simulation data with known direc-

tional connectivity pattern in different EEG frequency

bands. Also, the effects of signal-to-noise ratio (SNR) and

decimation were tested.

2 Methods and results

2.1 Simulation of the data

The connectivity data were simulated to test the perfor-

mance of the measures with a predefined causality structure

(Fig. 1). The simulation model design was inspired by the

model used in the study of Porcaro et al. [12] on functional

connectivity estimation, but with eight-node construction

here. Resting EEG (eyes closed) was applied in the simu-

lations as a source signal (S1, S2). EEG applied in the

simulation was measured from 25 persons with 20 elec-

trodes for 40 s with Fs = 1,000 Hz. For the simulation,

random sampling of the person, electrode and the data

segment (10,000 samples) was performed. Also, the noise

was regenerated for each repetition (N = 500). The con-

nectivity equations of the directional connectivity pattern

used are the following in function of sample time t:

x1ðtÞ ¼ S1ðtÞ þ 0:1 � x1ðt � 1Þ þ e1ðtÞx2ðtÞ ¼ x1ðt � 3Þ þ 0:1 � x2ðt � 1Þ þ e2ðtÞx3ðtÞ ¼ 2x1ðt � 6Þ þ 0:1 � x3ðt � 1Þ þ e3ðtÞ

x4ðtÞ¼x2ðt�12Þþx3ðt�10Þþ0:1 �x4ðt�1Þþ0:3 �e4ðtÞ

x5ðtÞ ¼ 3x3ðt � 5Þ þ 0:1 � x5ðt � 1Þ þ e5ðtÞx6ðtÞ ¼ 2x4ðt � 4Þ þ 0:1 � x6ðt � 1Þ þ e6ðtÞx7ðtÞ ¼ S2ðtÞ þ 0:1 � x7ðt � 1Þ þ e7ðtÞx8ðtÞ ¼ 2x7ðt � 6Þ þ 0:1 � x8ðt � 1Þ þ e8ðtÞ:

White Gaussian noise ei was added to the signal of each

node xi with SNR = [1, 5, 10]. A simulation (length 10,000

samples) was repeated 500 times for each SNR level.

Signals were decimated, resulting in typical sampling

frequency (Fs) 500 and 250 Hz, from the original Fs of

1,000 Hz. The power line artifact was removed from the

data with 50 Hz notch filter. Simulation and decimation

were performed with Matlab R2010a. Decimation was

implemented utilizing decimate-function, by first filtering

the signals with a phase neutral low pass Chebyshev type I

filter, with a cutoff frequency of 0.8 9 (Fs/2)/sampling

factor, and then re-sampling the smoothed signal at a

lower rate. The loss of detection power of the AR-based

connectivity measures during decimation is clearly

expected because the original model order 12 was chosen

based on the time delays in the original time series

(Fs = 1,000 Hz). The time delays in the decimated time

series cannot be the same and therefore the detection power

decreases. Thus, model orders 6 and 3 were also tested with

Fs = 500 Hz and Fs = 250 Hz, respectively.

2.2 Connectivity measures

In this study cross-correlation, magnitude-squared coher-

ence, PSI, DTF, PDC and normalized TE were applied to

the connectivity analysis of the simulated data. Mathe-

matical descriptions for the measures and further details are

presented in the Table 1 in the ‘‘Appendix’’. Detailed

descriptions are provided in the following articles: [11]

(cross-correlation and magnitude-squared coherence), [9]

(PSI), [6] (DTF), [2] (PDC) and [14] (TE).

Filtered signals were divided into segments of 4 s,

having a 50 % overlap. In case of PSI, different trials

(length = 10 s data/trial) were done consecutive for to

reach the statistical significance. The segments were nor-

malized to have zero mean and unit variance. For the

analysis performed with cross-correlation and TE the sig-

nals were also filtered with a finite impulse response (FIR)

Equiripple phase neutral filter (Astop 40–50 dB, Apass

0.1 dB, transition 1–2 Hz) to the following frequency

bands: 1–4 Hz (d), 4–8 Hz (h), 8–13 Hz (a), 13–30 Hz (b)

and 30–80 Hz (c). The analysis with other measures was

performed in the same frequency bands. Autoregressive

(AR) model order was defined to be the longest delay in the

connectivity pattern, i.e., 12.

The connectivity analyses were implemented with

Matlab R2010a. Following toolboxes were used in the

connectivity analyses: software to estimate the phase-slope

Fig. 1 The simulation scheme of the directional connectivity pattern.

S1 and S2 denote the source signals, X1–X8 the nodes of the pattern and

D the sample delay of each connection with Fs = 1,000 Hz

684 Med Biol Eng Comput (2012) 50:683–688

123

indexes [9, 10], BioSig software made for biosignal anal-

ysis [3] and TIM toolbox for the estimation of information-

theoretic measures from time series [5, 13].

2.3 Statistical analysis

Significant connections were determined by calculating a

distribution with the connectivity analyses of non-connected

white Gaussian noise signals (N = 4,000). The risk levels of

1–5 % (statistical thresholds) were determined from the

distribution. The results from the connectivity analyses (i.e.,

amount of direct causal connections vs. other relations

between the measures) were assessed by Chi-square tests.

The tests were performed with SPSS for Win 16.0.

2.4 Detecting the connections

The ability of the measures to distinguish the direct causal

connections from the non-causal connections was evaluated

with the simulated data (Fs = 1,000 Hz, SNR = 10) (Fig. 2).

Direct causal connection means that the direct link

between nodes was found, e.g., X1–X2 in Fig. 1. Non-

causal connection is either indirect causal connection (e.g.,

X1–X5 in Fig. 1) or false connection where there is no

connectivity between nodes (e.g., X4–X5 in Fig. 1).

All the measures were able to separate the direct causal

interactions from the non-causal relations in all frequencies

that are statistically significant with p \ 0.01. PDC detec-

ted less non-causal connections from all the measures.

Magnitude-squared coherence detected more non-causal

connections than any other measure. Cross-correlation

and PSI managed to detect almost all the direct causal

connections (99–100 %). DTF and TE, instead, missed

quite a lot of the direct causal connections (36–27 %).

Performance of coherence and PDC was equal when it

comes to the amount of detected direct causal connections;

they both detected 87 % of connections.

2.5 Effect of SNR

Reducing the SNR decreased the amount of the detected

direct causal connections while detected non-causal con-

nections either decreased or remained the same (Fig. 3).

Decreasing the SNR from 10 to 5 did not have a statisti-

cally significant (p \ 0.01) effect to the performance of the

measures. Instead, decreasing the SNR from 5 or 10 to 1

had a strong effect. Further, analysis showed that this

effect was strongest in the c-band. DTF and PDC tolerated

the low SNR better than the other measures. In case of

SNR = 1, PDC was already one of the most sensitive

connectivity measure by detecting more direct causal

connections than the other measures together with cross-

correlation and PSI. TE detected only 36 % of the direct

causal connections with SNR = 1, and missed the half of

the connections that it was able to detect with SNR = 10.

2.6 Effect of decimation

DTF and PDC detected more direct causal connections with

original Fs (1,000 Hz) than with the decimated data.

However, amount of detected non-causal connections was

reduced (PDC, DTF, and TE) with decimation (Fig. 4)

(model order 12). The effect increased when the sampling

factor increased, i.e., more detected direct causal connec-

tions were found with Fs = 500 Hz than with Fs = 250 Hz.

Decimation affected also the detected direct causal

connections found with TE. The effect was dependent on

frequency in the analysis with TE: higher Fs (250 to 500 to

1,000 Hz) resulted in significantly (p \ 0.01) less detected

direct causal connections in the d frequencies (602 to 557 to

Fig. 2 Percentages of the detected direct causal connections and

non-causal connections averaged over the frequency bands. CC cross-

correlation, Coh coherence. Perfect results would be 0 % of non-

causal connections and 100 % of direct causal connections

Fig. 3 The effect of SNR in the amount of detected non-causal

connections. Results are averaged over the frequency bands

Med Biol Eng Comput (2012) 50:683–688 685

123

468) and significantly (p \ 0.01) more detected direct

causal connections in the a (4 to 165 to 669), b (2 to 96 to

611) and c (3 to 1 to 104) frequencies. In h, changes were

508 to 681(significant, p \ 0.01) to 690 (insignificant),

respectively. The effect of decimation to the amount of

detected non-causal connections was similar as in case of

the direct causal connections for DTF, PDC and TE but the

strength of the effect was smaller (Fig. 4). With model

orders 6 and 3 (with Fs = 500 Hz and Fs = 250 Hz,

respectively), results were similar (not shown).

3 Conclusions

Directed connectivity measures with different theoretical

backgrounds, i.e., information theoretic, parametric-mod-

eling based or phase related, has earlier been compared in

this extend only in theory [11]. In the present work, we

compared a set of conventional and directed connectivity

measures with different theoretical backgrounds experi-

mentally with the simulation data based on resting eyes

closed EEG. We also explored the effect of noise level and

preprocessing (decimation). Main results were that all the

measures were able to separate direct causal interactions

from the non-causal relations in all frequencies. However,

PDC had the highest specificity and coherence the lowest

specificity of connections. Low SNR was tolerated better

with DTF and PDC than with the other measures. Deci-

mation affected the results of DTF, PDC and TE.

Coherence, as an undirected measure, detected the con-

nections equally between all the nodes that were somehow

connected, i.e., had the same source signal (S1 or S2); this

resulted in low specificity. PDC, instead, was able to dis-

tinguish the direct connections from the indirect ones, thus

PDC was found to have the highest specificity. The results

are consistent with the theoretical assumptions [7] as well as

with the previous directed connectivity studies made with

the measures based on the AR modeling, showing PDC to

be good direct connectivity estimator [1, 15, 16]. In our

study, cross-correlation and PSI were the most sensitive

measures by detecting over 10 % more of the direct causal

connections than the other measures. However, all the

measures had reasonable sensitivity, i.e., they detected the

direct causal connections within reasonable accuracy.

In this study, DTF and PDC tolerated the low SNR

better than other measures. This was expected as the SNR

has less influence on the AR-based measures, because

additive noise is a part of the underlying parametric model.

With SNR = 1, TE missed the half of the connections that

it was able to detect with SNR = 10. Decreasing the SNR

from 5 or 10 to 1 had a strong effect. This result was

consistent with the simulation study of Astolfi et al. [1],

where they compared the performance of DTF, direct DTF

and PDC on the connectivity analyses with the data sim-

ulated with neural mass model. They found no statistically

significant difference in the connectivity analysis made

with SNR = [10, 5, 3], but the increase in the error func-

tion was significant when SNR was reduced to 1.

Decimation affected the analysis made with DTF, PDC

and TE, but not the analysis made with cross-correlation,

coherence or PSI. This confirmed the findings in a recent

simulation study [4], in which the decimation was found to

lead to wrong conclusions in the analyses made with DTF and

PDC in certain circumstances. In their study, also high-pass,

low-pass and notch filtering of the signals were shown to

confuse the connectivity analyses made with DTF and PDC.

In addition, different effects of the decimation in the distinct

frequency bands are clear because the EEG has different

powers in the frequency bands. However, it remained unclear

why change of model order to keep the delay constant did not

ameliorated the results with AR-based measures.

First, limitation of the present study was that the simu-

lated connections were linear, while the activity of the

neural systems has a non-linear nature. However, the

applicability of the measures used in this study on non-

linear connectivity has been studied elsewhere [7, 9, 11].

Second, another question is how close to real data we can

reach with this kind of simulation data. However, in real

signals directional connectivity structure is typically not

known. To obtain the most reliable data structure, we

applied a real EEG signal to construct the simulated data

with predefined connections. Also, for future studies it is

suggested that different EEG signals, for example different

resting close eye EEG, resting open eye EEG, continuous

EEG oscillation in various tasks, and event-related EEG

need to be tested. Also, models with various node numbers,

connectivity relationships between nodes and time delays

need to be tested to run a comprehensive conclusion. For

Fig. 4 The effect of decimation in the amount of detected non-

causal connections. Results are averaged over the frequency bands

686 Med Biol Eng Comput (2012) 50:683–688

123

example, the effect of decimation may not be as obvious in

cases with longer delay. Furthermore, a separate estimation

for the correctness of the network should be performed in

future studies. Here, it was assumed that the number of

correctly detected connections reflects the performance

also in the network level.

This is the first experimental comparison of connectivity

measures with different theoretical backgrounds, i.e.,

information theoretic, parametric-modeling based or phase

related, in the directed connectivity analysis of the simu-

lated data. Parametric-modeling-based measures had the

highest sensitivity of connections and tolerance to SNR in

simulations based on resting closed eye EEG. However,

decimation of data has to be carefully considered with

these measures. It would be very interesting to apply these

measures to real intracranial EEG signals or EEG source

imaging results and it will be a subject in the further

studies.

Acknowledgments This work was supported by the Academy of

Finland.

Appendix

See Table 1.

Table 1 Mathematical description of the connectivity measures

Connectivity

measure

Equation

Cross-correlation The cross-correlation function Cxy between time-series signals x(t) and y(t) can be defined as

CxyðsÞ ¼PN�s

k¼1

xðk þ sÞy�ðkÞ ð1Þ

where N denotes the number of samples,s 2 �m;�mþ 1. . .m� 1;m½ � the time lag between the signals when m is the maximal time lag considered and *

the complex conjugate [5]. Maximum of the absolute values of the cross-correlation function describes the strength of the dependency, and the lag at

the corresponding value the time delay between the signals. In this study, the analyzed sequences were normalized so the autocorrelations at zero lag

are identically 1.0.

In the present study, m = 500 samples was applied

Coherence Magnitude-squared coherence can be defined as

cxyðf Þ ¼ Xðf ÞY�ðf Þh ij j2Xðf Þh ij j Yðf Þh ij j ð2Þ

where X(f) and Y(f) denote the Fourier transforms of time-series signals x(t) and y(t), respectively, Y* the complex conjugate of Y, �j j the magnitude and

�h i the average [5]

In the present study, Hamming window with length of 4,096 samples was applied

PSI Normalized PSI can be defined as

WXY ¼ =P

f2F

C�XY ðf ÞCXY ðf þ d f Þ !

=stdðWXY Þ ð3Þ

where CXY ðf Þ ¼ SXY ðf Þ=ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiSXXðf ÞSYY ðf Þ

pdenotes the complex coherency, S the cross-spectral matrix, df the frequency resolution, =ð�Þ the imaginary part,

F the set of frequencies over which the slope is summed and stdðWXY Þ is estimated by the jackknife method [9].

In the present study, a segment length of 2.5 s, epoch length of 5.0 s and number of epochs = 50 were applied in the computation of PSI

DTF DTF from node j to i can be defined as

c2ijðf Þ ¼

Hijðf Þj j2Pm

n¼1

Hinðf Þj j2ð4Þ

where H denotes the AR model’s transfer function and m the amount of variables [7].

In the present study model orders of 12, 6, and 3 were applied, as explained in the Sect. 2

PDC PDC from node j to i can be defined as

pijðf Þ ¼ Aijðf Þffiffiffiffiffiffiffiffiffiffiffiffiffiffia�j ðf Þajðf Þp ð5Þ

where A(f) denotes the Fourier transformed matrix including the parameters of the AR model, ajðf Þ the jth column of the matrix A, * the transposition and

complex conjugate operation [8, 16]. In the present study model orders of 12, 6, and 3 were applied, as explained in the section Sect. 2

TE Differential entropy of time-series signal x(t) can be defined as

HðXÞ ¼ �R

lðxÞ log½lðxÞ�dx ð6Þwhere l(x) denotes the probability density function of x [15].

Transfer entropy from time-series signal X(t) to Y(t) can be defined as

TEX!Y ¼ HðY ; YsÞ � HðYÞ½ �. . .

. . .� HðY ;Xs; YsÞ � HðYs;XsÞ½ �;ð7Þ

where H(*) denotes the (differential) (joint) entropy and * the past of the signals [6, 15].

Normalized difference between the two amounts of transfer information [from x(t) to y(t) and from y(t) to x(t)] can be defined as

nTEX!Y ¼ TEX!Y�TEY!X

TEX!YþTEY!Xð8Þ

Normalized transfer entropies were calculated according (8) for maximal transfer entropies computed according (7) with different lags (maximal

lag = 20 samples).

Med Biol Eng Comput (2012) 50:683–688 687

123

References

1. Astolfi L, Cincotti F, Mattia D, Lai M, Baccala L, Fallani F,

Salinari S, Ursino M, Zavaglia M, Babiloni F (2007) Comparison

of different cortical connectivity estimators for high-resolution

EEG recordings. Hum Brain Mapp 28:143–157

2. Baccala LA, Sameshima K (2001) Partial directed coherence: a

new concept in neural structure determination. Biol Cybern

84(6):463–474

3. BioSig, ‘‘BioSig—toolbox for biosignal analysis’’ (online).

Available: http://biosig.sourceforge.net/

4. Florin E, Gross J, Pfeifer J, Fink GR, Timmermann L (2010) The

effect of filtering on Granger causality based multivariate cau-

sality measures. NeuroImage 50(2):577–588

5. Gomez-Herrero G, Wu W, Rutanen K, Soriano M, Pipa G,

Vicente R (2010) Assessing coupling dynamics from an ensemble

of time series (online). Available: arXiv:1008.0539v1

6. Kaminski MJ, Blinowska KJ (1991) A new method of the

description of the information flow in the brain structures. Biol

Cybern 65(3):203–210

7. Kaminski M (2007) Multichannel data analysis in biomedical

research. In: Jirsa VK, McIntosh AR (eds) Handbook of brain

connectivity. Springer, Berlin, pp 327–355

8. Kus R, Kaminski M, Blinowska KJ (2004) Determination of EEG

activity propagation: pair-wise versus multichannel estimate.

IEEE Trans Biomed Eng 51(9):1501–1510

9. Nolte G, Ziehe A, Nikulin VV, Schlogl A, Kramer N, Brismar T,

Muller K-R (2008) Robustly estimating the flow direction of

information in complex physical systems. Phys Rev Lett 100(23):

234101

10. Nolte G, Ziehe A, Nikulin VV, Schlogl A, Kramer N, Brismar T,

Muller K-R, Phase-slope index—software (online). Available:

http://ml.cs.tu-berlin.de/causality/

11. Pereda E, Quiroga RQ, Bhattacharya J (2005) Nonlinear multi-

variate analysis of neurophysiological signals. Prog Neurobiol

77(1–2):1–37

12. Porcaro C, Zappasodi F, Rossini PM, Tecchio F (2009) Choice of

multivariate autoregressive model order affecting real network

functional connectivity estimate. Clin Neurophysiol 120:436–448

13. Rutanen K, Gomez-Herrero G, TIM-toolbox for estimation of

information-theoretic measures from time-series (online). Avail-

able: http://www.cs.tut.fi/*timhome/tim.htm

14. Schreiber T (2000) Measuring information transfer. Phys Rev

Lett 85:461–464

15. Velez-Perez H, Louis-Dorr V, Ranta R, Dufaut M (2008) Con-

nectivity estimation of three parametric methods on simulated

electroencephalogram signals. Conf Proc IEEE Eng Med Biol

Soc 2008:2606–2609

16. Winterhalder M, Schelter B, Hesse W, Schwab K, Leistritz L,

Timmer J, Witte H (2006) Detection of directed information flow

in biosignals. Biomed Tech (Berl) 51(5–6):281–287

688 Med Biol Eng Comput (2012) 50:683–688

123