STATISTICAL COMPLEXITY STATISTICAL COMPLEXITY ANALYSISANALYSIS

• Dr. Dmitry Nerukh• Giorgos Karvounis

What is Complexity?• Many different definitions.

• A natural system, is converted into a formal system that our mind can manipulate and we have a model.

• “Complexity is the property of a real world system that is manifest in the inability of any one formalism being adequate to capture all its properties…”

WHY USE COMPLEXITY ANALYSIS (1)

• When a new state of matter emerges from a phase transition,

• …certain “pattern formation” takes on the this “newness” with respect to other structures .

• This process is defined as “intrinsic” emergence.

• There is an increase in intrinsic computational capability which can be capitalised and measured.

WHY USE COMPLEXITY ANALYSIS (2)

• Contemporary physics can measure order (e.g. temperature) or ideal randomness (e.g. entropy, thermodynamics).

• No tools to address problems of innovation, or the discovery of patterns

• Measuring the computational capabilities of the system is the only way to address such questions:

• …discovering and quantifying emergence, pattern, information process and memory in quantitative units.

• The term intrinsic computation defines the way the system stores information with respect to time, transmits it between internal degrees of freedom and makes use of it in order to produce future behaviour.

METHODOLOGY (1) • Complexity estimates how sophisticated are the

dynamical laws governing the time evolution of the system.

• We adopted the approach by Crutchfield et. al. termed “computational mechanics”.

• We implement ideas from both Shannon entropy and KC algorithmic complexity theories, measuring the size of the informational description of the process.

• This is a direct approach to reveal the symmetries possessed by any spatial patterns and to estimate the “minimum amount of memory required to reproduce any configuration ensemble”.

……BUT WE ARE MODELLERS…BUT WE ARE MODELLERS…

…HOW DO WE DEAL WITH MODELS…??HOW DO WE DEAL WITH MODELS…??

METHODOLOGY (2)

• We can reconstruct an algorithmic machine (termed as “ε-machine”) that provides the means to build the statistically minimal “optimally predictive model ” .

• In order to build this machine, we need the smallest possible set of predictive states, the “causal states”.

• We can state that two predictive states are equivalent (~) if and only if they give rise to same future values in terms of conditional probabilities:

)|Pr()|Pr(: 21 pastfuturepastfuture StateCausal

COMPUTATIONAL IMPLEMENTATION(1) • The algorithm is based on the use of symbolic dynamics generated

from symbols assigned to discrete time steps.

• The crucial part in the implementation of the methodology is converting a continuous real signal into a sequence of symbols i.e “ signal symbolization” of the molecular trajectory. The one –dimensional case is shown below:

s0

s1

s1

s1

s0

s2

s1

t4

t3

t2

t0

sig

na

l

time

t1

s0

s2

s2

s2

s2

s2

s2

s2

s2

s0

s0

s0

s0

s0

CAUSAL STATE

Consider the following sequence:

bla.bla.bla.lab.lba.bla.bla.lab.bal.bla.alb.alb.bla.bla…

Bla . BlaPASTPAST FUTUREFUTURE

Alb . BlaPASTPAST FUTUREFUTURE

BlaAlb Bla CAUSALCAUSAL

statestate SScausalcausal

E-machine

• An -machine, the set of causal states and the probabilities of the transitions between them, provides a direct description of the patterns present in the system’s internal degrees of freedom.

E x am ple o f -m ach ine reconstruction : A and B a re tw o cau sa l sta tes o f the system .

N um bers on the a rrow s show s the transition p robab ility b etw een th e sta tes.

FINITE STATISTICAL COMPLEXITY

• Finite Statistical Complexity can be defined as “the minimum amount of evolutionary information (or hidden memory) required to statistically reproduce a process.”

•It expresses the informational size of the distribution of the causal states as measured by the Shannon Entropy:

).Pr(log)Pr( aSSC 2

•Statistical Complexity is based on the assumption that randomness is statistically simple: an ideal random process has zero statistical complexity. Equally, simple periodic processes give low complexity values as well.

•Complex process is the one that lies between these extremes and is an amalgam of predictable and stochastic mechanisms.

STATISTICAL COMPLEXITY OF A

ZWITTERION: a folding event

• We measured the statistical complexity of the dynamical trajectories of four significant atoms within a zwitterion.

• Attain insights regarding complexity and how this can be a useful tool to characterise or capture the folding event.

• Depending on the temperature of the simulation, the zwitterion adopts a stable folded conformation.

• Statistical Complexity Analysis of various atoms’ trajectories at the unfolded configuration and compare their values at the folded state.

COMPLEXITY ANALYSIS OF THE EXTENDED STATE

0 2000 4000 6000 8000 10000 12000

2

3

4

5

6

7

8

9

time (fs)

Dista

nce

(nm

)

Distance between N+andO-

UNFOLDED STATE

0 2000 4000 6000 8000 10000 120006.0

6.5

7.0

7.5

8.0

8.5

9.0

Com

plex

ity

time (fs)

N+

Cb/bone

O-

1

4000KUNFOLDEDSTATE

At the extended state, there is no significant change on the complexity value, as the zwitterion remains as an extended chain, following basically the same pattern throughout the process.

0 2000 4000 6000 8000 10000 120006.0

6.5

7.0

7.5

8.0

8.5

9.0Com

plex

ity

time (fs)

N+

O-

Cb/bone

4600K

FOLDED STATE

0 2000 4000 6000 8000 10000 12000

2

3

4

5

6

7

8

9Distance between N+and O-

Dista

nce

(nm)

time (fs)

FOLDED STATE

COMPLEXITY ANALYSIS OF THE FOLDED STATE

•In the folding event, there is a considerable drop in the complexity value, assigned to the transitional stage.

• Afterwards, there is a sudden rise in the complexity, until all atoms reach the same value, assigned to the pattern of the folded state.

COMPLEXITY ANALYSIS(1)

-4

-2

0

2

-2

0

2

4

2

4

Z

YX

COMPLEXITY ANALYSIS (2)

• The essentiality of complexity measurements is that we can distinguish those patterns in quantitative terms.

• Better insight to the mechanisms that underlie the formation of this structure and separate the more “ordered” regularities to those that are more “random”:

Ra nd o m ne ss 10

Sta

t. C

om

ple

xity

T RA N S I T I O N A LS T A GE

FO LDEDSTAG E

FUTURE WORK

• Further development of the algorithm in order to achieve a better representation of the -machine.

• Apply Statistical Complexity Analysis to a larger system such as protein folding and polymers’ phase transitions.

ACKNOWLEDGMENTS

For this work we are grateful to:

• Prof. R. Glen

• The Newton Trust and UNILEVER for their financial support.