Abstract: This overview of the free energy principle offers an account of embodied exchange with the world that associates conscious operations with actively inferring the causes of our sensations. Its agenda is to link formal (mathematical) descriptions of dynamical systems to a description of perception in terms of beliefs and goals. The argument has two parts: the first calls on the lawful dynamics of any (weakly mixing) ergodic system from a single cell organism to a human brain. These lawful dynamics suggest that (internal) states can be interpreted as modelling or predicting the (external) causes of sensory fluctuations. In other words, if a system exists, its internal states must encode probabilistic beliefs about external states. Heuristically, this means that if I exist (am) then I must have beliefs (think). The second part of the argument is that the only tenable beliefs I can entertain about myself are that I exist. This may seem rather obvious; however, if we associate existing with ergodicity, then (ergodic) systems that exist by predicting external states can only possess prior beliefs that their environment is predictable. It transpires that this is equivalent to believing that the world and the way it is sampled will resolve uncertainty about the causes of sensations. We will conclude by looking at the epistemic behaviour that emerges under these beliefs, using simulations of active inference.Key words: active inference autopoiesis cognitive dynamics free energy epistemic value self-organization.

I am therefore I think Karl Friston, University College London

Free Energy Workshop 2015

OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think

The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony

Active inferenceMinimizing expected uncertaintySaccadic searches and salience

Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision

Knowing your place Multi-agent gamesMorphogenesis

How can the events in space and time which take place within the spatial boundary of a living organism be accounted for by physics and chemistry? (Erwin Schrdinger 1943)

The Markov blanket as a statistical boundary (parents, children and parents of children)Internal states External statesSensory statesActive statesThe Markov blanket in biotic systems

Active states

External statesInternal statesSensory states

The Fokker-Planck equation

And its solution in terms of curl-free and divergence-free componentslemma: any (ergodic random) dynamical system (m) that possesses a Markov blanket will appear to engage in active inference

5But what about the Markov blanket?Reinforcement learning, optimal control and expected utility theory

Information theory and minimum redundancy

Self-organisation, synergetics and homoeostasis

Bayesian brain, active inference and predictive coding

Value

Surprise

Entropy

Model evidence

PavlovHakenHelmholtz

Barlow

PerceptionAction

OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think

The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony

Active inferenceMinimizing expected uncertaintySaccadic searches and salience

Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision

Knowing your place Multi-agent gamesMorphogenesis

Position

Simulations of a (prebiotic) primordial soupWeak electrochemical attractionStrong repulsionShort-range forces

ElementAdjacency matrix2040608010012020406080100120Markov BlanketHidden statesSensory statesActive statesInternal states

Markov Blanket = [B [eig(B) > ]]Markov blanket matrix: encoding the children, parents and parents of childrenFinding the (principal) Markov blanketADoes action maintain the structural and functional integrity of the Markov blanket (autopoiesis) ?

Do internal states appear to infer the hidden causes of sensory states (active inference) ?

Autopoiesis, oscillator death and simulated brain lesionsDecoding through the Markov blanket and simulated brain activation100200300400500-0.4-0.3-0.2-0.10TimeMotion of external stateTrue and predicted motion -505-8-6-4-20468PositionPositionPredictability2

TimeModesInternal states10020030040050051015202530

Christiaan Huygens

The existence of a Markov blanket necessarily implies a partition of states into internal states, their Markov blanket (sensory and active states) and external or hidden states.

Because active states change but are not changed by external states they minimize the entropy of internal states and their Markov blanket. This means action will appear to maintain the structural and functional integrity of the Markov blanket (autopoiesis).

Internal states appear to infer the hidden causes of sensory states (by maximizing Bayesian evidence) and influence those causes though action (active inference)

Interim summary

res extensa (extensive flow)res cogitans (beliefs)Belief productionFree energy functionalI am [ergodic] therefore I think

OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think

The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony

Active inferenceMinimizing expected uncertaintySaccadic searches and salience

Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision

Knowing your place Multi-agent gamesMorphogenesis

Objects are always imagined as being present in the field of vision as would have to be there in order to produce the same impression on the nervous mechanism - von Helmholtz

Thomas BayesGeoffrey HintonRichard FeynmanThe Helmholtz machine and the Bayesian brainRichard Gregory

Hermann von Helmholtz Objects are always imagined as being present in the field of vision as would have to be there in order to produce the same impression on the nervous mechanism - von Helmholtz Richard Gregory

Hermann von Helmholtz Impressions on the Markov blanket

Bayesian filtering and predictive coding

prediction update

prediction error

Making our own sensations

Changing sensationssensations predictionsPrediction errorChanging predictionsActionPerception

the

DescendingpredictionsAscending prediction errors

A simple hierarchy

whatwhereSensory fluctuations

Hierarchical generative models

frontal eye fieldsgeniculatevisual cortexretinal inputponsoculomotor signalsTop-down or backward predictionsBottom-up or forward prediction errorproprioceptive inputreflex arcPerception

David MumfordPredictive coding with reflexesAction

Prediction error (superficial pyramidal cells)Expectations (deep pyramidal cells)

Biological agents minimize their average surprise (entropy)

They minimize surprise by suppressing prediction error

Prediction error can be reduced by changing predictions (perception)

Prediction error can be reduced by changing sensations (action)

Perception entails recurrent message passing to optimize predictions

Action makes predictions come true (and minimizes surprise)

Interim summary

OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think

The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony

Active inferenceMinimizing expected uncertaintySaccadic searches and salience

Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision

Knowing your place Multi-agent gamesMorphogenesis

ThalamusArea X

Higher vocal centreHypoglossal Nucleus

creating your own sensations: Listening

time (sec)Frequency (Hz)sensations0.20.40.60.811.21.41.61.8250030003500400045005000

ThalamusArea X

Higher vocal centreHypoglossal Nucleus

creating your own sensations: SingingMotor commands (proprioceptive predictions)Corollary discharge(exteroceptive predictions)

time (sec)Frequency (Hz)sensations0.20.40.60.811.21.41.61.8250030003500400045005000

time (sec)Frequency (Hz)percept1234567250030003500400045005000012345678-50050100time (seconds)First level expectations (hidden states)012345678-40-20020406080time (seconds)Second level expectations (hidden states)Singing alone

time (sec)Frequency (Hz)percept1234567250030003500400045005000012345678-50050100time (seconds)First level expectations (hidden states)012345678-40-20020406080time (seconds)Second level expectations (hidden states)Singing together

-20-100102030405060-20-100102030405060Synchronizationsecond level expectations (first bird)second level expectations (second bird)-20-100102030405060-30-20-1001020304050No synchronizationsecond level expectations (first bird)second level expectations (second bird)Mutual prediction and synchronization of chaossynchronization manifold

OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think

The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony

Active inferenceMinimizing expected uncertaintySaccadic searches and salience

Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision

Knowing your place Multi-agent gamesMorphogenesis

saliencevisual inputstimulussamplingPerception as hypothesis testing saccades as experiments

Sampling the world to minimise expected uncertainty

I am [ergodic] therefore I think I think therefore I am [ergodic]

LikelihoodEmpirical priorsPrior beliefs

Frontal eye fields

Pulvinar salience mapFusiform (what)Superior colliculusVisual cortexoculomotor reflex arc

Parietal (where)

30

Visual samplesConditional expectations about hidden (visual) statesAnd corresponding perceptSaccadic eye movementsHidden (oculomotor) states

Saccadic fixation and salience maps

200400600800100012001400-202Action (EOG)time (ms)200400600800100012001400-505Posterior belieftime (ms)

vs.vs.

OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think

The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony

Active inferenceMinimizing expected uncertaintySaccadic searches and salience

Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision

Knowing your place Multi-agent gamesMorphogenesis

Generative modelsHidden states

Action

Control states

Continuous statesDiscrete statesBayesian filtering (predictive coding)Variational Bayes(belief updating)

Full priors control statesEmpirical priors hidden statesLikelihoodThe (normal form) generative modelHidden states

Action

Control states

KL or risk-sensitive control

In the absence of ambiguity:Expected utility theory

In the absence of uncertainty or risk:Bayesian surprise and Infomax

In the absence of prior beliefs about outcomes:

Prior beliefs about policiesQuality of a policy = (negative) Expected free energy

Extrinsic valueEpistemic value, information gain or reduction of expected uncertaintyPredicted divergenceExtrinsic valueBayesian surprisePredicted mutual information

midbrainmotor Cortexoccipital Cortexstriatum

Variational updatesPerception

Action selection

Incentive salienceFunctional anatomy

prefrontal Cortex

hippocampus

00.20.40.60.81020406080PerformancePrior preferencesuccess rate (%) FEKLRLDASimulated behaviour

Simulating conditioned responses and dopamine release (precision updates)

OverviewThe statistics of lifeMarkov blankets and ergodic systemsSimulations of a primordial soupI am therefore I think

The anatomy of inferenceGraphical models and predictive codingCanonical microcircuitsBirdsong and generalized synchrony

Active inferenceMinimizing expected uncertaintySaccadic searches and salience

Anatomy of choiceMarkov decision processesMinimizing expected free energyDopamine and precision

Knowing your place Multi-agent gamesMorphogenesis

Generative modelIntercellular signalingIntrinsic signalsExogenous signalsEndogenous signals

Knowing your place: multi-agent games and morphogenesis

-4-2024-4-3-2-101234Extracellular target signalposition

(Genetic) encoding of target morphologyPosition (place code) Signal expression (genetic code)

-4-2024-4-3-2-101234Target formposition

05101520253035-4-3-2-101234Morphogenesistimelocation-4-2024-4-3-2-101234Solutionlocation51015202530-420-400-380-360-340Free energytimeFree energy

Softmax expectationscellcell123456781234567851015202530-1.5-1-0.500.511.522.53Expectationstime51015202530-2-1.5-1-0.500.511.522.5timeAction

Morphogenesis

DysmorphogenesisGradientGradientIntrinsicExtrinsicExtrinsicTARGET

Regeneration of head05101520253035-4-3-2-101234morphogenesistimelocation-4-2024-4-3-2-101234location05101520253035-4-3-2-101234morphogenesistimelocationRegeneration of tailRegeneration

Each movement we make by which we alter the appearance of objects should be thought of as an experiment designed to test whether we have understood correctly the invariant relations of the phenomena before us, that is, their existence in definite spatial relations.

The Facts of Perception (1878) in The Selected Writings of Hermann von Helmholtz,Ed.R. Karl, Middletown: Wesleyan University Press, 1971 p. 384

Hermann von Helmholtz And thanks to collaborators:

Rick AdamsRyszard AuksztulewiczAndre BastosSven BestmannHarriet BrownJean DaunizeauMark EdwardsChris FrithThomas FitzGeraldXiaosi GuStefan KiebelJames KilnerChristoph MathysJrmie MattoutRosalyn MoranDimitri OgnibeneSasha Ondobaka Will PennyGiovanni PezzuloLisa Quattrocki KnightFrancesco Rigoli Klaas StephanPhilipp Schwartenbeck

And colleagues:

Micah AllenFelix BlankenburgAndy ClarkPeter DayanRay DolanAllan HobsonPaul FletcherPascal FriesGeoffrey HintonJames HopkinsJakob HohwyMateus JoffilyHenry KennedySimon McGregorRead MontagueTobias NolteAnil SethMark SolmsPaul Verschure

And many othersThank you