A New Kind of ScienceChapter 3

Matthew ZieglerCS 851 – Bio-Inspired Computing

2

Overview

• Wolfram’s scientific philosophy: The key to understanding complex behavior can be found in very simple discrete systems

• Elementary cellular automata

• Rule 110, a universal Turing machine

• Other systems in chapter 3

• Wolfram’s Observations and Conclusions

3

Simple Rules, Complex Behavior

• Overall theme is that systems with simple rules can produce complex behavior

– Is this counterintuitive? Wolfram thinks so.

• Many other systems have simple components but complicated behavior

– Nearly all topics on the course syllabus, e.g., neural nets, GAs, swarms

– Human brain, very complex considering only 12 million bytes in human genome*

– Many of these systems require additional inputs aside from initial conditions

• Wolfram considers systems with very simple rules and where output is based on initial conditions

R. Kurzweil, “Reflections on Stephen Wolfram's A New Kind of Science,” http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html

*

4

Elementary Cellular Automata

• 2^8 = 256 rules – Rigid array– Parallel update

5

Elementary Cellular Automata by the Numbers

• small persistent patterns (~170)

o stationary

o moving

• growing patterns (~85)

o repetitive patterns (~45)

o more complex patterns (36)

o fractal, nested patterns (24)

o random patterns (10)

o complex mixture of regular and irregular (1)

D. Drysdale, “Critical Review of A New Kind of Science,”http://freespace.virgin.net/david.drysdale/wolfram/review.html

6

Elementary Cellular Automata Taxonomy

• Class 1 – Repetition

• Class 2 – Nesting

• Class 3 – Randomness

• Class 4 – Localized Structures

R. Kurzweil, “Reflections on Stephen Wolfram's A New Kind of Science,” http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html

7

Class 1 – Repetition

• Degenerate, single color or checker board

• Nothing really interesting or surprising

Rule 250

8

Class 2 – Nesting

Rule 90

Repeated nested patterns

9

Class 3 – Randomness

Rule 30

• 10 rules of elementary CA are random

• Statistical analysis show randomness

• Are purely random patterns less complex than localized structures?

10

Class 4 – Localized Structures

Rule 110

• “Beyond randomness” P. 60

• Neither regular nor completely random

• Unique rule among 256 elementary CA

11

More on Rule 110

• Rule 110 displays complex behavior, despite simple rules and initial conditions

• Rule 110 discovered by Wolfram in the 1980s

– Wolfram proposed it was “Universal” e.g. could be used to simulate a universal Turing machine

• Proved by Matthew Cook (former Wolfram employee) in the1990s

– Wolfram suing Cook for presenting results before book released

• Throughout chapter 3 Wolfram demonstrates adding more complex rules do not give rise to more complex behavior

– However he does mention additional complexity is needed for system details

12

3 Color Cellular Automata

• More than 7 trillion rules

• Totalistic rules– New color from average of

previous colors – 6561 rules

• More complexity in rules

• Similar Behavior to elementary CA

13

Other Systems Surveyed in Chapter 3

• Mobile Automata

• Turing Machines

• Substitutional Systems

• Sequential Substitutional Systems

• Tag Systems

• Cyclic Tag Systems

• Register Machines

• Symbolic Systems

14

Mobile Automata

• Basic mobile automata does not show complex behavior

• Add another rule, update color of immediate neighbors

• Still 99% show repetitive behavior

– Every few thousands rules has nested structure

– Searching 50,000 rules randomness is found

• Random pattern and active cell movement

• Add a 2nd new rule, active cells can split or disappear

– Complexity now emerges

– Correlation b/tw number of active cells and complexity

• Activity is reason for more complex behavior in cellular automata than mobile automata

15

Turing Machines

• Behavior depends on state of the head and color of the

cell, not on the color of neighboring cells

• 2 states, 2 color Turing Machines show repetitive and

nested behavior

• 3 states: still only repetitive and nested behavior

• 4 states: random patterns emerge

• More than 4 states: randomness becomes more common,

but behaviors are not more complex

• Complexity threshold met at 4 states

16

Substitutional Systems

• Number of elements in a row can change

• Need rules that depend on both color of active element and a neighboring elements to get complexity above nesting

• Additional rule: allow elements to disappear, giving slow growth to the patterns

– Slow growth only exhibits repeating patterns

– Adding elements with 3 or more color gives complex random behavior

17

Sequential Substitutional Systems

• Similar to search and replace in text editor

• 1 or 2 replacements yield only simple patterns

• Complex patterns are found with 3 or more replacements

18

Tag Systems

• Fixed number of blocks removed at the beginning of the sequence, rules specify number and colors of block to be tagged onto the end of the sequence based on removed blocks

• 1 block removed yields repetitive or nested behavior

• 2 block removed produces complex behavior

19

Cyclic Tag Systems

• Very simple rules

• 1 underlying rule with two cases, alternate between cases at each step

• Fluctuations in growth of the sequence are many times random

20

Register Machines

• 2 register machine, 2 different instructions– increment– decrement-jump

• 4 or less instructions in program: repetitive behavior

• 5-7 instructions in program: nested behavior

• 8 instructions in program: random behavior when a register value turns to zero

• 3 register machine, random at 7 instructions in a program, but no more complexity

• Adding different kinds of instructions also does not increase complexity

21

One Instruction Computer

subz A,B,C

mem[A] = mem[A] - mem[B]; if (mem[A] == 0)

PC = C;

else PC = PC + 1;

22

Symbolic Systems

• Simple symbolic rules, such as: e[x_][y_] -> x[x[y]]

• This particular rule eventually always stabilizes

• But simple symbolic rules can give produce repetitive, nested, and random patterns

23

Wolfram’s Observations

• Are there special features inherent with CA that are crucial to complexity?

– Rigid array counterexample: Substitutional Systems

– Parallel update counterexample: Mobile Automata

• None of the features in CA actually matter: Complexity is universal

24

Complexity Threshold

• Adding complexity to rules increases complexity of behavior until complexity threshold is crossed

• Complexity threshold is low

• Adding more complex rules past the threshold does not perceivably increase complexity of behavior

complexity

Repetition

Nesting

Complexity Threshold

25

References and Links

• Collection of Reviews– http://www.math.usf.edu/~eclark/ANKOS_reviews.html

• Critical Review of  "A New Kind of Science" and Notes on "A New Kind of Science"– http://freespace.virgin.net/david.drysdale/wolfram/review.html

• Reflections on Stephen Wolfram's "A New Kind of Science" – http://www.kurzweilai.net/meme/frame.html?main=/articles/art0464.html

• One Instruction Set Computer – http://www.cs.ucsd.edu/classes/fa96/cse30/lec2/oic.html

• Life Applet– http://hensel.lifepatterns.net/