Feynman Machine https://arxiv.org/abs/1609.03971

New Neural Architectures for Cortical and Machine Intelligence

Eric Laukien, Richard Crowder and Fergal ByrneOgma Intelligent Systems Corp Ogma

Properties of Interacting Nonlinear Dynamical Systems

Since Lorenz (1963) discovered his famous butterfly attractor, Applied Mathematicians have studied the properties of Nonlinear Dynami-cal Systems using numerical meth-ods. Unlike fixed-point (equilibrium) and periodic systems, chaotic sys-tems’ states contain perfect informa-tion about their future. Takens (1981) proved that a signal from a NDS also contains this information, allowing an observing system to reconstruct its dynamics, forecast its future, analyse causality, and control it.

Evidence of Feynman Machine in Primate NeocortexTajima, Yanagawa, Fujii & Toyoizumi, 2015Untangling Brain-Wide Dynamics in Consciousness by Cross-EmbeddingPLOS Computational BiologyThe Feynman Machine is a network

of communicating, adaptive NDSs, which collaborate to emergently form predictive models of the sensorimo-tor world. Based on our systems neuroscience theory, artificial Feynman Machine networks are composed of local-ly coupled encoder-decoder pairs. The bidirectional hierarchy process-es spatiotemporal data over multiple timescales (see Github link below for source code).

Feynman Machine in Memcomputing & Neuromorphic Hardware

L6 - Input/Motor Control, Simulation, Attention

output to higher region

L4 - Driven Dynamical Modeller

L1 - Feedback Integration

feedback tolow

er regions

from

thal

amus

L2/3 - Temporal Pooling/CharacterisationRepresenting Nonstationarity

feedback from higher region

L5 - Motor/Behaviour

motor +

thalamus

from

low

er re

gion

attention, gating of inputs from

thalamus

Multilayer Model of aNeocortical Region

… …

Encoder 1 Decoder 1

Decoder 2

Encoder N

sensory predictioncurre

nt a

ctio

ns

Decoder N

Encoder 2chosen actionse

nsor

y in

put

encoder output

optional top-down supervising signal

1

1

1

encoder output

decoder error

decoder error

decoder error

Predictor 1

Predictor 2

Predictor Ncorrected prediction

corrected prediction

corrected prediction

encoder output

Feynman Machine:Structure and Function

Nonlinear Dynamical Systems + Mesoscale Neuroscience + Machine Learning

Fact 1: Nonlinear Dynamical Systems’ Signals Contain Complete Information

Fact 2: Coupled NDSs Exhibit Emergent Complex Behaviour

Fact 3: Brains are Nested Networks of Nonlinear Circuits

Idea: Build a Network from Learning NDS Modules - the Feynman Machine

https://ogma.ai https://github.com/ogmacorp eric@ogmacorp.com rcrowder@ogmacorp.com fergal@ogmacorp.com

Traversa & di Ventra (2015) showed networks of memelements have the power of Nondeterministic Turing Machines. Current work concerns combining Takens-based signalling and local prediction learning to this regime. A demo FM has been built on a small FPGA. We are currently ex-ploring opportunities to base FMs on memcomputing and neuromorphic hardware. ReferencesHamilton, F., Berry, T. and Sauer, T., 2016. Kalman-Takens filtering in the presence of dynamical noise. arXiv preprint arXiv:1611.05414.Lorenz, E. N. 1963. Deterministic nonperiodic flow. Journal of the Atmospheric Scienc-es 20(2):130–141.Tajima, S.; Yanagawa, T.; Fujii, N.; and Toyoizumi, T. 2015. Untangling brain-wide dy-namics in consciousness by cross-embedding. PLoS Comput Biol 11(11):e1004537.Takens, F. 1981. Detecting strange attractors in turbulence. Springer.Traversa, F. L., and Di Ventra, M. 2015b. Universal memcomputing machines. IEEE transactions on neural networks and learning systems 26(11):2702–2715.

Tajima et al. (2015) used Takens’ Theo-rem to demonstrate causal hierarchy in conscious primate neocortex.

Applications of the Feynman Machine

The artificial Feynman Machine (left) is based on an abstraction of our 2015 model of regions of neocortex (above).

The latest “Exponential Memory” ver-sion of the OgmaNeo Feynman Ma-chine library has higher layers strid-ing over multiple timesteps and providing contextual feedback down-wards.

The Noisy Lorenz Attractor (above) is a challenging task for time series learning. The Feynman Machine con-verges in hundreds of steps to er-rors comparable with Hamilton et al (2016).