chaos and information dr. tom longshaw spsi sector, dera malvern [email protected]
TRANSCRIPT
2
Some background information DERA is an agency of the MoD Employs over 8000 scientists Over 30 sites around the country Largest research organisation in
Europe
SPSI Sector Parallel and Distributed Simulation
3
Introduction A further example of chaos
When is a system stable? Measuring chaos
Energy, entropy and information. Avoiding chaos when not wanted
How to avoid chaotic programs! Practical applications of chaos
What use can chaos be?
4
Further reading
ChaosChaos: Making a new Science, James Gleick, Cardinal(Penguin), London 1987.http://www.around.com
Chaos and EntropyThe Quark and the Jaguar, Murray Gell-Mann, Little, Brown and Company, London 1994.
ComplexityComplexity, M. Mitchell Waldrop, Penguin, London, 1992.
http://www.santafe.eduhttp://www-chaos.umd.edu/intro.html
5
When is a system stable? A street has 16 houses in it, each
house paints its front door red or green.
Each year each resident chooses a another house at random and paints their door the same colour as that door.
Initially there are 8 red and 8 green. Is this system stable…?
6
What controls the chaos? If we increase the size of the
population (number of houses) does the system become more stable?
If we increase the sample size (e.g. look at 3 of our neighbours) does the system become more stable?
7
Sample results
Varying the population size
0
10
20
30
40
50
1 2 3 4 5 6 7 8 9 10
Population size(log 2)
Tim
e to
con
verg
e
1 sample
8
02468
10
1 2 3 4 5
Sample size
Con
verg
ence
tim
e
Population=16
Varying the sample size
9
Varying both together
1 3 5 7 9
9S4
0
10
20
30
40
50
Convergence time
Population (log)
Sample size
0
5
10
15
20
25
30
35
40
45
1 2 3 4 5 6 7 8 9 10
10
Why is the system unstable?
11
Potential Energy
The “potential to change”
0
10
20
30
40
50
60
70
80
Number red
Ener
gyPhase change
Initial state
12
Landscapes of possibility Watersheds ...
13
Chaos and Entropy
Chaos and entropy are synonymous.
Entropy was originally developed to describe the chaos in chemical and physical systems.
In recent years entropy has been used to describe the ratio between information and data size.
14
Information
Measuring the ratio of information to bits.
00000000000000000000000000 Low (0) information
01010101010101010101010101 Little information
01011011101111011111011111 More information
01101001110110000101101110 Random (0 information!)
15
Measuring information
Shannon entropy (1949) The ability to predict based on an
observed sample. Algorithmic Information Content
(Kolmogorov 1960) The size of program required to
generate the sample Lempel-Ziv-Welch (1977,1984)
The zip it and see approach!
16
When is a system stable? When there is insufficient energy in
the system for the system to change its current behaviour.
Paradoxically such systems are rarely interesting or useful.
Entropy
Info
rmat
ion Complexity
Simplicity Total randomness
17
Characteristics of a chaotic system Unpredictability Non-linear performance Small changes in the initial settings
give large changes in outcome The butterfly effect
Elegant degradation Increased control increases the
variation
18
What makes a chaotic system? Non-Markovian behaviour. Positive feedback: state(n) affects
state(n+1). Any evolving solution. Simplicity of rules, complex
systems are rarely chaotic, just unpredictable.
Complex systems often hide simple chaotic systems inside.
19
Dealing with chaos
Avoid programming with integers! Avoid “while” loops Add damping factors
Observers and pre-conditions Add randomness into your
programs!
20
Practical Applications
“Modern” economic theory [Brian Arthur 1990]
Interesting images and games Fractals, SimCity, Creatures II
Genetic algorithms Advanced Information Systems “Immersive simulations”
21
Information Systems
Conventional database store data in a orderly fashion.
Reducing the data to its information content increases the complexity of the structure…
… but it can be accessed much faster, and some queries can be greatly optimised.
22
smallWorlds
Developed to model political and economic situations
Difficult to quantify Uses fuzzy logic and tight
feedback loopsIf demand is high then price increases.If price is high then retailers grow.If supply is high then price decreases.If price is low then retailers shrink.
23
Conclusions
Computer scientists should recognise chaotic situations.
Chaos can be avoided or forestalled.
Chaos is not always “bad”. Sometimes a chaotic system is
better than the alternatives.