nonlinear time series analysis with the tisean package · 2008-05-26 · nonlinear time series...

Nonlinear time series analysis with the TISEAN package

Eckehard Olbrich

MPI MIS Leipzig

Potsdam SS 2008

Olbrich (Leipzig) 23.05.2008 1 / 16

Deterministic chaos

Deterministic chaos: Aperiodic irregular temporal behaviour inlow-dimensional deterministic systems

Nonlinear time series analysis: A battery of methods to characterize,predict and model time series as nonlinear deterministicsystems instead as linear stochastic systems as in the“classical” time series analysis

Successful applications: Electric circuits, Laser, Hydrodynamicexperiments (Taylor–Couette flow), ...

Unsuccessful attempts: Climate data, EEG data, finance data, ...


The henon map

The logistic map:xn+1 = 1 − ax2

n

Add a second degree of freedom in order to make it invertible:

xn+1 = 1 − ax2n + byn

yn+1 = xn

Example: 10 000 data points with henon - delay plot with delay

-1.5

-1

-0.5

0

0.5

1

1.5

0 20 40 60 80 100

x n

n

-1.5

-1

-0.5

0

0.5

1

1.5

-1.5 -1 -0.5 0 0.5 1 1.5

x n+

1

xnOlbrich (Leipzig) 23.05.2008 3 / 16

Embedding - False nearest neighbours with false nearest

Delay vector: xxxn = (xn, xn−1, xn−2, . . . , xn−m+1)

False nearest neighbour: next neighbour in m dimensions, distanceincreases by more than the factor f in m + 1 dimensions

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1 2 3 4 5 6 7 8 9 10

frac

tion

of fa

lse

near

est n

eigh

bour

s

embedding dimension m

f=2f=3


Estimating entropies with boxcount

Block entropies Hm(ǫ)

0

2

4

6

8

10

12

0.001 0.01 0.1 1

H(m

,ε)

ε

log(N)1.6-1.25*log(ε)


Estimating entropies with boxcount

Conditional entropies hm(ǫ) = Hm(ǫ) − Hm+1(ǫ)

0

0.5

1

1.5

2

2.5

3

3.5

4

0.001 0.01 0.1 1

h(m

,ε)

ε

λ=0.4190.82-log(ε)


Correlation sum and dimension - d2

Time series {x(t0), x(t0 + ∆), . . . , x(t0 + N∆)}Delay embedding xxx = (xt , xt−d∆, . . . , xt−(m−1)d∆)

Correlation sum

C (m, ǫ) =2

(N − m) · (N − m − 1)

N−m∑

i=1

N∑

j=i+1

Θ(ǫ − ||~Xi − ~Xj ||)

∝ ǫD2

Correlation dimension D2

D2 = limǫ→0

D2(m, ǫ) = limǫ→0

d log C (m, ǫ)

d log ǫ


Correlation sum and dimension - d2

1e-05

1e-04

0.001

0.01

0.1

1

0.001 0.01 0.1 1

C(m

,ε)

ε

0

0.5

1

1.5

2

2.5

3

3.5

4

0.001 0.01 0.1 1

D(m

,ε)

ε

DKS=1.25

correlation sum correlation dimension


Lyapunov exponents

Estimating the largest Lyapunovexponent using lyap k

-8

-7

-6

-5

-4

-3

-2

-1

0

0 5 10 15 20

S(n

)

n

0.425*x-6.95

Stretching factor vs. time steps fordifferent neighborhood sizesλmax = 0.425

Lyapunov exponents from thesystem equations:

λ1 = 0.419, λ2 = −1.62,DKY = 1.26

Lyapunov exponents from fittingthe Jacobian with lyap spec:

λ1 = 0.41, λ2 = −1.55,DKY = 1.27


Flow data — the Lorenz system

Lorenz equations:

x = s(−x + y)

y = −xz + rx − y

z = xy − bz

Integrated with lorenz at stan-dard parameters

s = 10

r = 28

b = 8/3

-20

-15

-10

-5

0

5

10

15

20

0 1000 2000 3000 4000 5000

t

-20 -15 -10 -5 0 5 10 15 20-25-20

-15-10

-5 0

5 10

15 20

25 5

10 15 20 25 30 35 40 45

z(t)

x(t)

y(t)

z(t)


Embedding — finding the optimal delay time

Delay embedding of the x-component

-20-15

-10-5

0 5

10 15

20 -20-15

-10-5

0 5

10 15

20

-20-15-10-5 0 5

10 15 20

x(t+2∆)

x(t) x(t+∆)

x(t+2∆)

-20-15

-10-5

0 5

10 15

20 -20-15

-10-5

0 5

10 15

20

-20-15-10-5 0 5

10 15 20

x(t+20∆)

x(t) x(t+10∆)

x(t+20∆)

-20-15

-10-5

0 5

10 15

20 -20-15

-10-5

0 5

10 15

20

-20-15-10-5 0 5

10 15 20

x(t+36∆)

x(t) x(t+18∆)

x(t+36∆)

-20-15

-10-5

0 5

10 15

20 -20-15

-10-5

0 5

10 15

20

-20-15-10-5 0 5

10 15 20

x(t+100∆)

x(t)

x(t+50∆)

x(t+100∆)



Rule of thumb: First zero of the autocorrelation function (corr) or firstminimum of the mutual information function (mutual)

0

0.5

1

1.5

2

0 100 200 300 400 500

τ

Correlation functionMutual information

0

0.5

1

1.5

2

0 10 20 30 40 50

τ

Correlation functionMutual information

First zero of the autocorrelation function at 314 time steps. First minimumof the mutual information function at 18 time steps.



-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

x(t+

18∆)

x(t)

-20

-15

-10

-5

0

5

10

15

20

-20 -15 -10 -5 0 5 10 15 20

x(t+

314∆

)x(t)

Mutual information Correlation functiond = 18 d = 314


What is an optimal state space?

One possibility - maximize the scaling range in dimension estimates.Let us consider the correlation dimension:

C (ǫ) =2

N(N − 1)

N∑

i=1

N∑

j=i+1

Θ(ǫ − ||xxx i − xxx j ||)

Scaling range ǫl ≤ ǫ ≤ ǫuFor m > D0

C (ǫ) ∝ ǫD2

Lower end of the scaling range in the noise free case:

C (ǫl) =2

N(N − 1)

Olbrich (Leipzig) 23.05.2008 10 / 16

Estimating the scaling range

What about the upper bound?Correlation sum approximates Renyi entropies of second order

− log Cm(ǫ) ≈ H(q=2)m (ǫ)

For q = 1 we know Hm(ǫ) ≥ mhKS . Assuming the same for q = 2 andcalling the corresponding entropy hC one gets

−logCm(ǫu) ≥ mhC

−logCm(ǫl) ≥ mhC − D2 logǫlǫu

Olbrich (Leipzig) 23.05.2008 11 / 16

Estimating the scaling range

What about the upper bound?

−logCm(ǫl) ≥ mhC − D2 logǫlǫu

and therefore

ǫuǫl

≤(

e−mhCN2

2

)1/D2

.

In order to get a scaling in one decade of ǫ:

N ≥√

2 · 10D2/2 · emhC /2

For D2 = 4, m = 5 and hC = ln 2 we would get N > 4525.

Olbrich (Leipzig) 23.05.2008 11 / 16

Maximizing the scaling range

Maximizing the scaling range ⇒ Minimizing H(ǫu).

Hm(ǫu) = mhm−1(ǫ) +m−1∑

n=1

nδhn(ǫ)

with δhn(ǫ) = hn−1(ǫ) − hn(ǫ) and h0(ǫ) = H1(ǫ). In a first approximation

Hm(ǫu) = mhKS + MI MI = limǫ→0

MI (ǫ)

with the mutual information MI = δh1. For different delays τ we have

hKS = τ hKS

thus minimizing Hm(ǫ) leads to

dMI (τ)

dτ

∣

∣

∣

∣

τ=τopt

= −mhKSd2MI (τ)

dτ2

∣

∣

∣

∣

τ=τopt

> 0

For hKS = 0 this gives the first minimum of the mutual information.Olbrich (Leipzig) 23.05.2008 12 / 16

Correlation dimension for different delays

0

0.5

1

1.5

2

2.5

0.1 1 10

D2(

ε)

ε

delay d=1

0

0.5

1

1.5

2

2.5

0.1 1 10

D2(

ε)

ε

Delay d=10

0

0.5

1

1.5

2

2.5

3

3.5

0.1 1 10

D2(

ε)

ε

Delay d=18

0

2

4

6

8

10

12

0.1 1 10

D2(

ε)

ε

Delay d=50

Olbrich (Leipzig) 23.05.2008 13 / 16

Temporal correlations — the Theiler correction

For finite data temporal correlations might lead to a based estimate of thecorrelation sum. Theiler correction: remove W temporal neighbours fromthe correlation sum

C (ǫ) =2

N − W (N − W − 1)

N∑

i=1

N∑

j=i+W+1

Θ(ǫ − ||xxx i − xxx j ||)

Space-time separation plot: plots cumulative distribution for distances ǫconditioned on the temporal differences.Space-time separation plot (stp) for the Lorenz system (m=3,d=18)

0

5

10

15

20

25

30

35

40

0 50 100 150 200 250 300 350 400 450 500

ε

∆t

Olbrich (Leipzig) 23.05.2008 14 / 16

EEG — spurious dimension due to temporal correlations

1

10

100

1000

0 10 20 30 40 50 60 70 80 90 100

ε

t [∆t]

0

5

10

15

20

10 100

D2(

m,ε

)

ε

tW=0tW=3

tW=601.64

Space-time separation plot Correlation dimension

Olbrich (Leipzig) 23.05.2008 15 / 16

Prediction and Modeling

xxxn+1 = FFF (xxxn+1)

which reduces toxn+1 = F (xn, . . . , xn−m+1)

in the case of delay embedding.

Local methods: Basic idea: Looking for similar events in the past andusing their future for prediction.Local constant (lzo-run): Using the average as prediction.Local linear (lfo-run): Fitting a linear model for similarevents, i.e. neighbors in phase space.

Global models: Parameterizing the function FFF and fitting the parameters.Polynomials (polynom)Radial basis functions (rbf) — related to neural networks(not included in TISEAN)

Olbrich (Leipzig) 23.05.2008 16 / 16

Prediction and Modeling

-30

-25

-20

-15

-10

-5

0

5

10

15

20

0 100 200 300 400 500 600 700 800 900 1000

x(t)

t

originallocal constant prediction

local linear predictionglobal model rbf

global model polynom

Relative forecast errors:

lfo 1.12e-04poly 8.28e-04lzo 1.25e-02rbf 4.03e-02

Polynomial fit used 4th order polynomials.

Olbrich (Leipzig) 23.05.2008 16 / 16

nonlinear time series analysis with the tisean package · 2008-05-26 · nonlinear time series...

Documents