life under the lens christos h. papadimitriou. alan m. turing (1912 – 1954) started cs in 1936 on...

LifeUnder

the Lens

Christos H. Papadimitriou

Alan M. Turing (1912 – 1954)

• Started CS in 1936• On the right foot

(universality)• Pioneering work in

brain science,morphogenesis, statistics, analysis,phyllotaxis,…

We’ve come a long way

Compilers Operating systems

Databases

Chips Machine learning

Graphics

Networks

Slick interfaces

Global information environment

Moore’s law

fast algorithms

And yet the biggest problem of all remains unsolved…

Is P = NP?“Can exhaustive search

always be curtailed?”

Soooo, Computer Science

• Creator and curator of an essential technology

• Fountainhead of one of the most fundamental questions in all of science

• What else can we do?

A Lens on the Sciences?

Computation is everywhere!

Wielding the Lens

The computational worldview provides new insights into,

and teststhe most prestigious theories about the universe

Statistical Physics and Algorithms

How does the lake freeze?

The mysteryof phasetransitionsvs. the convergence of algorithms

Quantum computation:Turning a question on its head

“Oh my God, how do you simulate such a system on a computer?”

Richard Feynman, 1983

“But what if we built a computer out of these things?”

“Quantum computation is as much about testing Quantum Physics as it is about building powerful computers.”

Umesh Vazirani

Equilibria: Behavior predictions in Economics

They exist in two-player

zero-sum games,1928

John Nash 1950:

all games have one!

The Story of Equilibria

John Forbes Nash, Jr.1928 - 2015

They exist in two-player

zero-sum games,1928

John Nash 1950:

all games have one!

The Story of Equilibria It’s just fixedpoints, isn’t it?

In markets too!Price equilibria (Arrow-Debreu 1954)

“The Nash equilibrium lies at the foundations of modern economic thought.”

Roger Myerson

Universality of equilibria

• Nash’s Theorem estabishes it for Nash equilibrium, the Arrow-Debreu Theorem for price equilibria, …

• Key desideratum of any solution concept

“Nobody would take seriously a solution concept that is void for some games.”

Roger Myerson

Surprise!

Finding

equilibria

is an

intractable

problem!

And intractability meansthat universality is suspect

Nash equilibria cannot be a useful prediction for the behavior of a group of people

“If your laptop can’t find it, neither can the market”

Kamal Jain

Evolution under the Lens

The Origin of Species

• Possibly the world’s most masterfully compelling scientific argument

• Natural selection• Common ancestry

Evolution Theory since 1859

• Genetics (Mendel, 1866 – really, 1901)• The crisis (1901 – 1930)• The synthesis through math (1930 – 1960)• The genomics revolution (1980 – )

Brilliant theory, a deluge of data -- and yet most important questions

unanswered

• Why so much genetic diversity?• What is the role of sex/recombination?• Is Evolution optimizing something?• How do complex adaptations happen?

To begin, on the role of sex

• Fisher-Muller theory• Muller’s ratchet• The parasite theory• ……

• Each needs special assumptions to work,

none is credible as the answer

Sex as randomization

• Recall the power of Randomization in computation

• Sampling: can predict properties of astronomically large distributions

• This is what sex enables:

“How well would this new allele perform

with all possible genetic combinations?”

Surprising connection with game theory

• Evolution of a population under weak selection can be seen as a repeated game between genes:

• The strategies of each gene are its alleles• The common utility is the organism’s fitness• The genes update their probabilities of play

through multiplicative updates• Each allele’s multiplier ~ its mixability

Multiplicative updates!

• A simple, common-sense algorithm known in CS for its surprising aptness at solving many sophisticated problems

• At each step, increase the weight of allele i by a factor of (1 + fi)

• fi is the allele’s fitness in the current environment created by the other genes -- i.e. its mixability [Livnat et al. 2007]

Multiplicative updates:Dual interpretation

• Convex optimization duality: Each gene “seeks to optimize” the sum of two quantities:

allele frequencies cumulative fitness

maxx Φ(x) = H(x) + s F(x)

entropy selection strength

Recall the Big Questions

• Why so much genetic diversity?• What is the role of sex/recombination?• Is Evolution optimizing something?• How do complex adaptations happen?

Finally: The frontier within

Study of the Brain:

• Babies vs computers• Clever algorithms vs what happens in

cortex• Understanding Brain anatomy and function

vs understanding the emergence of the Mind• No known “neurally plausible” algorithm

solving a nontrivial problem• Plus, disruptive insights: downward traffic,

reciprocity and clustering, prediction,…

The Great Disconnects

Les Valiant on the Brain [1995 - ]

• Theory is essential• The neuroidal model

and vicinal algorithms• Items and operations on items

Neuroids and Synapses

…

Wj = Σi fi wij > Tj fj =1

(pj, Tj) Φ(fj, pj, Tj)

(qij,wij) Ψ(qij,wij,pj,Wj)

pi, Ti, fi

pj, Tj, fj

qij,wij

DGn,p

Valiant’s Vicinal Algorithms

• A conservative, formal model of cortical computation

• Please: minimal control,

synchrony, awkwardness,

cleverness,

Items

• An item = a set of r neurons• Representation of real-world idea, e.g.

“almond”• Simultaneous firing of these neurons is

coterminal with thinking of this idea• Theorem [Valiant]: Reasonable values of r

are compatible with what we know about the Brain.

Operations on Items

BA

Join(A, B)

Operations on Items: Link

HALink(A, H)

But are these realistic?And what are they good for?

Theorem [Valiant] Join and Link can be implemented by vicinal algorithms whp in two steps.

Using Join and Link, one can learn patterns of legth two

of length n > 2?

BA

PJoin(A, B)

Predictive Join [P., Vempala 2015 COLT]

Neurorealistic?

Theorem: PJoin can be implemented by a vicinal algorithm whp in three steps.

Useful? Unsupervised Learning

“Learn a pattern x”

=

“on presentation of x,

create a top-level item I(x), which

will not participate in further PJoins, and

will fire precisely on all subsequent presentations of x”

Vicinal unsupervised learning by PJoin

Algorithm:

While a pattern is presented for time > Tits input neurons fire with probability pnew PJoins are formed with prob q

after a retraction period Rand existing PJoins do their thing

Vicinal unsupervised learning by PJoin

01 0 01

Second presentation

01 0 01

Other patterns: Share and build

00 1 01

Unsupervised Learning

Theorem: Any m patterns in {0, 1}n can be learned whp and with total height O(log m + log n), provided that

T ≥ log n / p, and

D ≥ log n

Simulations

• Patterns with n = 100• all learning activity completed in < 80 steps• sharing as predicted• majority of firing traffic downwards

A tantalizing question

What if the Mind emerges from

Rudimentary primitives

+

Massive hardware

+

“Organic” environments

?

Next Challenges

• Invariants: Clustering-cum-interpolation• Language: A “last-minute adaptation”• Hypothesis: it evolved so as to exploit the

Brain’s strengths• NB: PJoin and “Plink” are ideally fitted for

learning grammar

Sooooo… a prediction

Understanding scientific problems

in terms of algorithmic ideas

will be in the future as widespread

-- and as productive --

as formulating scientific problems

in terms of equations

has been in the past

Thank You!