suggested reading relevant journal articles -...
TRANSCRIPT
NONLINEAR DYNAMICS AND CHAOS
Patrick E McSharry
Systems Analysis, Modelling & Prediction Group
www.eng.ox.ac.uk/samp
Tel: +44 20 8123 1574
Trinity Term 2007, Weeks 3 and 4
Mondays, Wednesdays & Fridays 09:00 - 11:00
Seminar Room 2
Mathematical Institute
University of Oxford
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.1
LECTURE 1: INTRODUCTION
1. Introduction
2. Maps
3. Flows
4. Fractals and Attractors
5. Bifurcations
6. Quantifying Chaos
7. Nonlinear Time Series Analysis
8. Nonlinear Modelling and Forecasting
9. Real-World Applications
10. Weather Forecasting
11. Biomedical Models
12. Time Series Analysis Workshop
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.2
Suggested Reading
Strogatz, S. H., Nonlinear Dynamics and Chaos: With Applications to Physics, Biology,Chemistry, and Engineering, Reading, MA: Addison-Wesley (1994)
Eubank, S., and D. Farmer, An introduction to chaos and randomness. In Jen, E. (Ed.), 1989Lectures in Complex Systems. Santa Fe Institute Studies in the Sciences of Complexity,Lecture Vol. II, pp. 75-190. Reading, MA: Addison-Wesley, (1990)
Ott, E. and Sauer, T. and Yorke, J. Coping with Chaos, J. A. John Wiley & Sons, New York(1984)
Ott, E., Chaos in Dynamical Systems, Cambridge: Cambridge University Press (1993)
Schuster, H.G., Deterministic Chaos: An Introduction, VCH, (1995).
Wiggins, S., Introduction to Applied Nonlinear Dynamical Systems and Chaos, Springer(1990)
Kantz, H. and Schreiber, T., Nonlinear Time Series Analysis, Cambridge Univ. Press (1997)
Abarbanel, H.D.I, Analysis of Observed Chaotic Data, Springer, (1996)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.3
Relevant journal articles
Yorke, J. and Li, T. Y., Period Three Implies Chaos, American Mathematical Monthly82:985-992 (1975)
May, R., Simple mathematical models with very complicated dynamics. Nature 261:459-467 (1976)
Packard, N. and Crutchfield, J. and Farmer, J. D. and Shaw, R., Geometry from a timeseries, Phys. Rev. Lett. 45: 712-716 (1980)
Crutchfield, J. P, N. H. Packard, J. D. Farmer, and R. S. Shaw. (1986) Chaos, ScientificAmerican 255:46-57 (1996)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.4
Sources of information
Journals:Physica DPhysical Review LettersPhysical Review EPhysics Letters A
International Journal of Bifurcation and Chaos
Online discussion groups:
sci.nonlinearwww.jiscmail.ac.uk/lists/allstat.html
www.jiscmail.ac.uk/lists/timeseries.html
Websites:www.societyforchaostheory.org (Society for chaos theory)
www.physionet.org (MIT-Harvard biomedical database and tools)
www.comdig.org (Complexity Digest)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.5
Suggested special topics
Look for evidence of low-dimensionality in real time series
Investigate the predictability of a real time series using nonlinear methods
Available time series:electronic circuitslaserssunspot record
electricity data (demand, price, grid frequency)
weather data (temperature, precipitation, wind speed)
economic data (GDP, inflation, interest rates, unemployment)
financial data (stockmarket tick data, S&P500, Dow Jones, FTSE100)
biomedical signals (ECG, EEG, blood pressure, respiration, blood gases)
Construct and investigate a nonlinear model of a particular system
Compare nonlinear methods for forecasting, (e.g. RBFs vs. local linear)
Develop new methods for classification of health/disease using biomedical signals
TISEAN package
Matlab software
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.6
Special topic instructions
Should be one week of work
Marking scheme:-
content 20 ptspresentation 5 pts
Two copies of final report required
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.7
Linear analysis
Definition of linearity:
L(ax) = aL(x)
L(x + y) = L(x) + L(y)
Principle of superposition: If x and y are solutions, then z = ax + by is also a solution
Advantages of linear models:
Often have analytical solutions
A large body of historical knowledge for helping with model specification and estimation
Less parameters - smaller chance of overfitting
Can employ Fourier spectral analysis and associated techniques
Disadvantages of linear models:
Real-world systems are usually nonlinear
Linearity is a first order approximation and neglects higher orders
Stanislaw Ulam: nonlinear science is like non-elephant zoology
In practice, while underlying dynamics may be nonlinear, observed data may only providesufficient resolution for linear models
Need relevant null hypothesis tests for nonlinearity (surrogate data)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.8
Normal distributions
Mean x0 and variance σ2:
p(x) =1√
2πσ2exp
»−(x − x0)2
2σ2
–
All higher order moments are given in terms of x0 and σ
Easily manipulated: if x ∼ N(0, σ2x) and y ∼ N(0, σ2
y), then x + y ∼ N(0, σ2x + σ2
y)
Central limit theorem: sum of a large number of IID random variables (with finite mean andvariance) is normally distributed
For linear models:Normal distributions are preserved by principle of superposition
Normally distributed forecast errors: Maximum likelihood gives least squares
Useful for calculating prediction intervals
Problems for nonlinear systems:
Use of normal distributions neglects possibility of asymmetric distributions
Fat tailed distributions imply larger probability of worse case scenarios (riskmanagement)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.9
Dynamical systems I
Variables Linear Nonlinear
n=1 Growth, decay, equilibrium
Exponential growth Fixed points
RC circuit Bifurcations
Radioactive decay Overdamped systems, relaxational dynamics
Logistic equation for single species
n=2 Oscillations
Linear oscillator Pendulum
Mass and spring Anharmonic oscillator
RLC circuit Limit cycles
2-body problem Biological oscillators
Predator-prey cycles
Nonlinear electronics (van Der Pol)
n ≥ 3 Chaos
Civil engineering Strange attractors (Lorenz)
Electrical engineering 3-body problem (Poincar e)
Chemical kinetics
Iterated maps (Feigenbaum)
Fractals (Mandelbrot)
Forced nonlinear oscillators (Levison, Smale)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.10
Dynamical systems II
Variables Linear Nonlinear
n À 1 Collective phenomena
Coupled harmonic oscillators Coupled nonlinear oscillators
Solid-state physics Lasers, nonlinear optics
Molecular dynamics Nonequilibrium statistical mechanics
Equilibrium statistical mechanics Nonlinear solid-state physics
Heart cell synchronisation
Neural networks
Economics
Continuum Waves and patterns Spatio-temporal complexity
Elasticity Nonlinear waves (shocks, solitons)
Wave equations Plasmas
Electromagnetism (Maxwell) Earthquakes
Quantum mechanics (Schr odinger, Heisenberg) General relativity (Einstein)
Heat and diffusion Quantum field theory
Acoustics Reaction-diffusion, biological waves
Viscous fluids Fibrillation
Epilepsy
Turbulent fluids
Life
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.11
Napolean’s Army’s Russian Campaign
Map drawn by the French engineer Charles Joseph Minard in 1861 to show the tremendouslosses of Napolean’s army during his Russian Campaign of 1812
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.12
Happiness time evolution
From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.13
Happiness versus GDP
From Culture and Subjective Well-being, edited by Ed Diener & Eunkook M. Suh (2002)Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.14
Time series from nonlinear systems
A
B
C
D
E
F
G
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.15
Mathematical characterisation
Time series: variables are recorded as a function of time
Steady state: a constant solution of a mathematical equatione.g. Homeostasis: relative constancy of the internal environment with respect to variablessuch as blood sugar, blood gases, blood pressure and pH. Control mechanisms constrainvariables to narrow limits. e.g. Following a hemorrhage, reflex mechanisms quickly restoreblood pressure to equilibrium values.
Oscillations: periodic solutions of mathematical equationse.g. Heartbeat, respiration, sleep-wake cycles and reproduction
Irregular activity: intrinsic fluctuations, can even be present when external parameters arerelatively constant. Two distinct mathematical descriptions: noise and chaos
Noise:Variability cannot be linked with any underlying stationary or periodic process
e.g. Fluctuating environment: eating, exercise, rest and posture affects heart rate, bloodpressure, blood-sugar levels and insulin levels
e.g. Respiratory Sinus Arrhythmia (RSA): heart rate increases during inspiration
Chaos:Irregularity that arises in a deterministic system
Chaos can exist without influence of external noise
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.16
Noise versus Chaos
Inter-event time sequences (e.g. heartbeat, neurons firing)
Believed to be a random process
Simplest model for such a random process is a Poisson process:
Probability of event to occur in a very short time increment dt is Rdt
The probability R is independent of the previous history
Probability of two or more events occurring during dt is negligible
Probability of k events in time interval t is Pk(t) =(Rt)k
k!e−Rt (Poisson distribution)
Probability that interval between event and (k + 1)st following event is
pk(t) =R(Rt)k
k!e−Rt (PDF of Poisson process)
Average time between events is 1/R and variance is 1/R2
Nonlinear map, ti+1 = −(1/R) ln |1 − 2 exp(−Rti)| also gives p(t) = Re−Rt
Observation of an exponential probability density is not sufficient to identify a Poissonprocess!
Use recurrence plots (e.g. ti+1 versus ti) to identify structural equations
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.17
An exponential distribution, but not a Poisson process
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1
1.5
2
ln2/3 ti
ti+1
0 10 20 30 40 50 60 70 80 90 1000
0.2
0.4
0.6
0.8
1
1.2
1.4
i
ti
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.5
1
1.5
2
2.5
3
3.5
4
t
p(t)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.18
Determinism and Predictability
After Newton people believed in a deterministic, and hence, predictable Universe“Given for one instant an intelligence which could comprehend all the forces by which natureis animated and the respective situation of the beings who compose it—an intelligence andsufficiently vast to submit these data to analysis—it would embrace in the same formula themovements of the greatest bodies of the universe and those of the lightest atom; for it,nothing would be uncertain and the future, as the past, would be present before its eyes.”[P.S. Laplace, 1814]
“Laplacian dream” excludes stochastic laws of physics
Laplace acknowledged that we would never achieve the “intelligence” required—a tacitappreciation that deterministic systems might not, in practice, be predictable
deterministic 6= predictable
Laplace saw probabilities as a way to describe our ignorance of a deterministic system
Analytic expediency means most mathematics revolves around linear systems
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.19
Poincaré
After tackling the 3-body problem Poincarè identified the phenomenon of sensitivedependence on initial conditions (SDIC), this provided a definition of “chaos”
“If we knew exactly the laws of nature and the situation of the universe at the initial moment,we could predict exactly the situation of that same universe at a succeeding moment. Buteven if it were the case that the natural laws had no longer any secret for us, we could stillonly know the initial situation approximately. If that enabled us to predict the succeedingsituation with the same approximation, that is all we require, and we should say that thephenomenon had been predicted, that is is governed by laws. But it is not always so; it mayhappen that small differences in the initial conditions produce very great ones in the finalphenomena. A small error in the former will produce an enormous error in the latter.Prediction becomes impossible, and we have the fortuitous phenomenon.”[H. Poincaré,1903]
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.20
Lorenz’s butterfly effect
0 1 2 3 4 5 6 7 8−20
−15
−10
−5
0
5
10
15
20
time [secs]
x(t)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.21
Sensitive dependence on initial condition
−20 −15 −10 −5 0 5 10 15 200
5
10
15
20
25
30
35
40
45
50
x
z
Perfect model and perfect knowledge of observational uncertainty
Predictability varies with position
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.22
Chaos
Advent of digital computer allowed numerical investigation of nonlinear equations
Lorenz found SDIC in a numerical model of the atmosphere and constructed the “Lorenzsystem” to illustrate the effect in a simple system [1963]
Yorke and Li coined the word “chaos” in 1975
May demonstrates chaos in the one-dimensional Logistic map in 1976
Chaos becomes trendy, “Chaos” is published by James Gleick, 1987
Claims of chaos in the brain, heart, economy, stockmarket, ...
Investigations of nonlinear dynamical systems, claims of chaos are played down!
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.23
Two faces of Chaos
The word chaos refers to disorder and extreme confusion
To a scientist, it implies “deterministic disorder”
This might suggest that a chaotic system should be unpredictable
On the contrary, a chaotic deterministic system is, in principle, perfectly predictable
The sensitive dependence of the system dynamics to the initial conditions (SDIC) impliesthat, in reality, any error in specifying the initial condition will lead to an erroneous prediction
Laplace suggests using probabilistic predictions to overcome the problems, of divergingtrajectories, posed by chaotic systems
Chaos is sometimes used as a scapegoat: meteorologists blame chaos for inaccuratepredictions when it is often model inadequacy that is at fault
Important research: separating model inadequacy (structural and parametrical errors) fromeffects of observational uncertainty
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.24
Deterministic versus Stochastic
stochastic systemxt+1 = axt + εt εt : N (0, 1)
deterministic systemyt+1 = ayt + σzt
zt+1 = 4zt(1 − zt)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.25
Deterministic versus Stochastic
Deterministic : Everything is described by a single point in state space. This descriptioncompletely determines the future.
Stochastic : Knowledge of present states does not determine the evolution of future states.
Can you tell if a system is stochastic or deterministic?
Should a system be modelled as stochastic or deterministic?
High dimensional deterministic chaotic system might be modelled as a stochastic system
State space trajectory of an autonomous, deterministic system never crosses itself
Sources of forecast uncertainty:Uncertainty in the initial condition
Model inadequacy (parametrical and structural)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.26
Introduction to Dynamical Systems
A state is an array of numbers that provides sufficient information to describe the futureevolution of the system.
If m numbers are required, then these form an m-dimensional state vector x.
The collection of these state vectors defines an m-dimensional state space.
The rule for evolving from one state to another may be expressed as a discrete map or acontinuous flow:
map xt+1 = F (xt)
flow x(t) = f(x(t))
Fixed point of a map: x0 = F (x0)
Fixed point of a flow: x0 = f(x0) = 0
Non-autonomous system: x = f(x, t)
Autonomous system: x = f(x)
If the non-autonomous nature is due to periodic terms it can be made autonomous
Dissipative flow: ∇ · f < 0 implies contracting state space volume
Hamiltonian systems are non-dissipative or conservative, preserving state space volume(Liouville theorem)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.27
Properties of dynamical systems
Determinism: trajectories should not diverge when going forward in time
Invertibility: A dynamical system is invertible if each state x(t) has a unique predecessorx(t − 1). This implies that trajectories should never merge.
Thus continuous deterministic flows are always invertible!
Maps derived from flows (Poincaré maps) are also invertible
Reversibility: if the dynamical system obtained by the transformation t → −t is equivalent tothe original one
Invariance under coordinate transforms: an invariant of a dynamical system represents afundamental property of that dynamical system, e.g. dimensions and Lyapunov exponents
System invariant offer a means of summarising the behaviour of a particular system: (e.g.health and disease)
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.28
Simple Pendulum
An example of a simple two-dimensional dynamical system
From Newtons’s second law, knowledge of the forces, position and velocity are sufficient todetermine future motion
Pendulum (constrained to move in the plane)
Dynamics fully specified by the displacement angle θ(t) and the angular velocity θ(t)
State vector given by x(t) = [θ(t), θ(t)]
Let m be the mass of the pendulum
g is the acceleration due to gravity
l is the length of the pendulum
Tangential restoring force due to gravity: −mg sin θ
Tangential force due to angular acceleration: mlθ
In the absence of friction, dynamics are governed by
d
dtθ = θ
d
dtθ = −g
lsin θ
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.29
Fixed Points
Consider the fixed point of a flow f(x0) = 0
Let x(t) = x0 + ε(t)
ε = f(x0 + ε)
ε = f(x0) + Dxf(x0)ε + O(||ε||2)
ε ≈ Dxf(x0)ε = Jε
J is Jacobian matrix of partial derivatives
The solution isε(t) = eJtε0
Let λi be (distinct) eigenvalues of J
P−1JP = Λ
where Λii = λi and Λij = 0 if i 6= j
Let ε = Py so
y = eΛty0
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.30
Classification of fixed points
<(λi) 6= 0 for all i: hyperbolic fixed point
Otherwise non-hyperbolic fixed point
Hyperbolic fixed points:
sinks: all <(λi) < 0
sources: all <(λi) > 0
saddle point: some positive and some negative
If =(λi) = 0: node
If =(λi) 6= 0: focus
Non-hyperbolic fixed points:
if <(λi) > 0 for some i: unstable
some <(λi) < 0 some = 0: neutrally stable
all <(λi) = 0: centre
Nonlinear dynamics and chaos c© 2007 Patrick McSharry – p.31