sharpening occam ’s razor with quantum mechanics

Sharpening Occam’s razor with Quantum

MechanicsSISSA Journal Club

Matteo Marcuzzi 8th April, 2011

Niclas Koppernigck(Copernicus)

Clausius Ptolemaeus(Ptolemy)

Tyge Brahe(Tychonis)

Describing Systems

Describing Systems

Johannes Kepler

Describing Systems

.3

2

cRT

Describing Systems

Algorithmic Abstraction

212

21

rmmGF

Describing Systems

Algorithmic AbstractionSame output

Describing Systems

Same outputDifferent intrinsic information!

Solar system

celestial objects

Sun FlaresPlanet Orography

MeteorologyPeople behaviour

Compton Scattering

Describing Systems

Same outputDifferent intrinsic information!

Much more memory

required!OCCAM’S RAZOR

Describing SystemsN Spin Chain

Up parity

1 spin-flip per second10 if even

if odd

0

Describing SystemsN Spin Chain

Up parity10 if even

if odd

0

1 spin-flip per second

1

Describing SystemsN Spin Chain

Up parity10 if even

if odd

0

1 spin-flip per second

1 0

Describing SystemsN Spin Chain

Up parity10 if even

if odd

0

1 spin-flip per second

1 0 1

Describing SystemsN Spin Chain

Up parity10 if even

if odd

0

1 spin-flip per second

1 0 1 0

Describing SystemsN Spin Chain

Up parity10 if even

if odd

0

1 spin-flip per second

1 0 1 0 1

Describing SystemsN Spin Chain

Up parity10 if even

if odd

0

1 spin-flip per second

1 0 1 0 1 0

Describing SystemsN Spin Chain

Up parity10 if even

if odd

0

1 spin-flip per second

1 0 1 0 1 0 1

Describing SystemsN Spin Chain

Up parity10 if even

if odd

0

1 spin-flip per second

1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

N bits needed

Describing SystemsHidden System

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

read x

return (x+1) mod 21-bit only!

Statistically equivalent output

N bits

Computational Mechanics

• Statistical equivalence

•Measure of complexity

• Pattern identification

0m

Computational Mechanics

0m 0stm

• Statistical equivalence

•Measure of complexity

• Pattern identification

Computational Mechanics

0m 0stm 04 m

• Statistical equivalence

•Measure of complexity

• Pattern identification

Computational Mechanics

0m 0stm 04 m ?

• Statistical equivalence

•Measure of complexity

• Pattern identification

Computational Mechanics

�

21012 SSSSSS Stochastic Process

DiscreteStationary

iSRandom Variables A: Alphabet

edcbaA ,,,,

1,0'A

GCTAA ,,,''

Computational Mechanics

�

21012 SSSSSS Stochastic Process

DiscreteStationary

iSRandom Variables A: Alphabet

123 SSSS �

Pasts

210 SSSS Futures

Computational Mechanics

�

21012 SSSSSS Stochastic Process

DiscreteStationary

iSRandom Variables A: Alphabet

123 ssssA ��

Set of histories

210 ssssA Set of future strings

Computational Mechanics

�

21012 SSSSSS Stochastic Process

DiscreteStationary

Machine

000101000101110101101…

Statistical Equivalence

)()( sSsS MS

� PP

1,0A

Computational Mechanics

�

21012 SSSSSS Stochastic Process

DiscreteStationary

Machine

A�

1,0A…010100010

…1100111

…01010101

States

Partition R

Computational Mechanics

�

21012 SSSSSS Stochastic Process

DiscreteStationary

Machine

A�

States

Partition R

1R

2R

ijS RsRas ���P

a

Computational Mechanics

�

21012 SSSSSS Stochastic Process

DiscreteStationary

Machine

A�

1R

2R ijM

aij RRaT ,)( P Transition Rates

)(11

aT

)(12

aT

Rj

Ri

Computational Mechanics

�

21012 SSSSSS Stochastic Process

DiscreteStationary

OCCAM POOL

Computational MechanicsA little information theory

spspSHAs

log Shannon entropy

XSH Conditional entropy YXSH ,

XSHSHXSI : Mutual information

SSIE�

: Excess entropy

SH

Computational Mechanics

Machine Cannot distinguish between them

SSHSH�

R

Partition RA�

We want to preserve information

SSHSH�

R

Computational Mechanics

Machine

SSHSH�

R

Partition RA�

We want to preserve information

SSHSH�

R

with the least possible memory

0C Log(# states)minimize

Computational Mechanics

Machine

SSHSH�

R

Partition RA�

We want to preserve information

SSHSH�

R

with the least possible memory

minimize RHC Statisticalcomplexity

Computational Mechanics

SSHSH�

R

We want to preserve information

SSHSH�

R

with the least possible memory

minimize RHC Statisticalcomplexity

OCCAM POOL Optimal partition

Computational Mechanics

Optimal partition

We want to preserve informationwith the least possible memory

minimize RHC Statisticalcomplexity

SSHSH�

R

)'()( ssss SS

�� �� PP

ε-machine

ε

'~ ss ��if

Causal States

(unique)

Computational Mechanics: Examples

2-periodic sequence

2-periodic, ends with

2-periodic, ends with

1p1p

A

B

I

initial state

Computational Mechanics: Examples

2-periodic sequence

1p1p

A

B

I

initial state21p

21precurrent transient

Computational Mechanics: Examples1D Ising model

p p

p

p

ijV transfer matrix

jij

i Vuujip

Computational Mechanics: Examples1D Next-nearest-neighbours Ising

p

p

p

p

p

p p

p

31 J

12 J

2.0T

2

Computational Mechanics: Examples1D Next-nearest-neighbours Ising

p

p

p

p

p

p p

p

31 J

12 J

2.0T

23

Computational Mechanics: Examples1D Next-nearest-neighbours Ising

p

p

p

p

p

p p

p

31 J

12 J

2.0T

23

1

Computational Mechanics: Examples1D Next-nearest-neighbours Ising

p

p

p

p

p

p p

p

31 J

12 J

2.0T

23

1

Computational Mechanics: Examples1D Next-nearest-neighbours Ising

p

p

p

p

p

p p

p

31 J

12 J

2.0T

negligible

1

1

8B

Computational Mechanics: Examples1D Next-nearest-neighbours Ising

p

p

31 J

12 J

2.0T

1

1

period 3

period 18B

Sharpening the razor with QM

EC Statistical complexit

y

Excess entropy SSI

�: RH

EC Ideal system

Sharpening the razor with QM

,,AA�

ε

ε-machines are deterministic

ε

,,AA�

ε

Sharpening the razor with QM

1R

3R2R

4R

0,, )(44

)(34

)(24 blueblueblue TTT EC

,,AA�

ε

Sharpening the razor with QM

EC 0)( cijT

fixed i,c unique j

fixed j,c unique i

ideal

Sharpening the razor with QM

ε qεcausal state Ri system state i

symbol “s” symbol state s

siTSis

skik

,

)()(sijT

q-machine states

qεsystem state i

symbol state s

siTSis

skik

,

)(

q-machine states

Sharpening the razor with QM

CLASSICAL

QUANTUM

Prepare kS

Measure C.S.j t

2kSjt )(t

kjTProbability

tjS

)( iiq RS PP ip

qεsystem state i

symbol state s

siTSis

skik

,

)(

q-machine states

Sharpening the razor with QM

CLASSICAL

QUANTUM )( iiq RS PP ip

ii

i ppC log

logtrCq

iii

i SSp

qCC

E

E

qεsystem state i

symbol state s

siTSis

skik

,

)(

q-machine states

Sharpening the razor with QM

CLASSICAL

QUANTUM

ii

i ppC log

logtrCq

iii

i SSp

qCC

ijji SS

E

E

ks

sjk

sik TT

,

)()(Ideal system

E

E

qεsystem state i

symbol state s

siTSis

skik

,

)(

q-machine states

Sharpening the razor with QM

CLASSICAL

QUANTUM

qCC

ijji SS

Non-ideal systems

Quantum mechanics improves efficiency

Sharpening the razor with QM

single spinp

21

p

p

p

p1 p1 21

p

?21

21

C

E

qC

References

M. Gu, K. Wiesner, E. Rieper & V. Vedral - "Sharpening Occam's razor with Quantum Mechanics" - arXiv: quant-ph/1102.1994v2 (2011)

C. R. Shalizi & J. P. Crutchfield - "Computational Mechanics: Pattern and Prediction, Structure and Simplicity" - arXiv: cond-mat/990717v2 (2008)

D. P. Feldman & J. P. Crutchfield - "Discovering Noncritical Organization: Statistical Mechanical, Information Theoretic, and Computational Views of Patterns in One-Dimensional Spin Systems" - Santa Fe Institute Working Paper 98-04-026 (1998)