life under the lens christos h. papadimitriou. alan m. turing (1912 – 1954) started cs in 1936 on...
TRANSCRIPT
LifeUnder
the Lens
Christos H. Papadimitriou
Alan M. Turing (1912 – 1954)
• Started CS in 1936• On the right foot
(universality)• Pioneering work in
brain science,morphogenesis, statistics, analysis,phyllotaxis,…
We’ve come a long way
Compilers Operating systems
Databases
Chips Machine learning
Graphics
Networks
Slick interfaces
Global information environment
Moore’s law
fast algorithms
And yet the biggest problem of all remains unsolved…
Is P = NP?“Can exhaustive search
always be curtailed?”
Soooo, Computer Science
• Creator and curator of an essential technology
• Fountainhead of one of the most fundamental questions in all of science
• What else can we do?
A Lens on the Sciences?
Computation is everywhere!
Wielding the Lens
The computational worldview provides new insights into,
and teststhe most prestigious theories about the universe
Statistical Physics and Algorithms
How does the lake freeze?
The mysteryof phasetransitionsvs. the convergence of algorithms
Quantum computation:Turning a question on its head
“Oh my God, how do you simulate such a system on a computer?”
Richard Feynman, 1983
“But what if we built a computer out of these things?”
“Quantum computation is as much about testing Quantum Physics as it is about building powerful computers.”
Umesh Vazirani
Equilibria: Behavior predictions in Economics
They exist in two-player
zero-sum games,1928
John Nash 1950:
all games have one!
The Story of Equilibria
John Forbes Nash, Jr.1928 - 2015
They exist in two-player
zero-sum games,1928
John Nash 1950:
all games have one!
The Story of Equilibria It’s just fixedpoints, isn’t it?
In markets too!Price equilibria (Arrow-Debreu 1954)
“The Nash equilibrium lies at the foundations of modern economic thought.”
Roger Myerson
Universality of equilibria
• Nash’s Theorem estabishes it for Nash equilibrium, the Arrow-Debreu Theorem for price equilibria, …
• Key desideratum of any solution concept
“Nobody would take seriously a solution concept that is void for some games.”
Roger Myerson
Surprise!
Finding
equilibria
is an
intractable
problem!
And intractability meansthat universality is suspect
Nash equilibria cannot be a useful prediction for the behavior of a group of people
“If your laptop can’t find it, neither can the market”
Kamal Jain
Evolution under the Lens
The Origin of Species
• Possibly the world’s most masterfully compelling scientific argument
• Natural selection• Common ancestry
Evolution Theory since 1859
• Genetics (Mendel, 1866 – really, 1901)• The crisis (1901 – 1930)• The synthesis through math (1930 – 1960)• The genomics revolution (1980 – )
Brilliant theory, a deluge of data -- and yet most important questions
unanswered
• Why so much genetic diversity?• What is the role of sex/recombination?• Is Evolution optimizing something?• How do complex adaptations happen?
To begin, on the role of sex
• Fisher-Muller theory• Muller’s ratchet• The parasite theory• ……
• Each needs special assumptions to work,
none is credible as the answer
Sex as randomization
• Recall the power of Randomization in computation
• Sampling: can predict properties of astronomically large distributions
• This is what sex enables:
“How well would this new allele perform
with all possible genetic combinations?”
Surprising connection with game theory
• Evolution of a population under weak selection can be seen as a repeated game between genes:
• The strategies of each gene are its alleles• The common utility is the organism’s fitness• The genes update their probabilities of play
through multiplicative updates• Each allele’s multiplier ~ its mixability
Multiplicative updates!
• A simple, common-sense algorithm known in CS for its surprising aptness at solving many sophisticated problems
• At each step, increase the weight of allele i by a factor of (1 + fi)
• fi is the allele’s fitness in the current environment created by the other genes -- i.e. its mixability [Livnat et al. 2007]
Multiplicative updates:Dual interpretation
• Convex optimization duality: Each gene “seeks to optimize” the sum of two quantities:
allele frequencies cumulative fitness
maxx Φ(x) = H(x) + s F(x)
entropy selection strength
Recall the Big Questions
• Why so much genetic diversity?• What is the role of sex/recombination?• Is Evolution optimizing something?• How do complex adaptations happen?
Finally: The frontier within
Study of the Brain:
• Babies vs computers• Clever algorithms vs what happens in
cortex• Understanding Brain anatomy and function
vs understanding the emergence of the Mind• No known “neurally plausible” algorithm
solving a nontrivial problem• Plus, disruptive insights: downward traffic,
reciprocity and clustering, prediction,…
The Great Disconnects
Les Valiant on the Brain [1995 - ]
• Theory is essential• The neuroidal model
and vicinal algorithms• Items and operations on items
Neuroids and Synapses
…
Wj = Σi fi wij > Tj fj =1
(pj, Tj) Φ(fj, pj, Tj)
(qij,wij) Ψ(qij,wij,pj,Wj)
pi, Ti, fi
pj, Tj, fj
qij,wij
DGn,p
Valiant’s Vicinal Algorithms
• A conservative, formal model of cortical computation
• Please: minimal control,
synchrony, awkwardness,
cleverness,
Items
• An item = a set of r neurons• Representation of real-world idea, e.g.
“almond”• Simultaneous firing of these neurons is
coterminal with thinking of this idea• Theorem [Valiant]: Reasonable values of r
are compatible with what we know about the Brain.
Operations on Items
BA
Join(A, B)
Operations on Items: Link
HALink(A, H)
But are these realistic?And what are they good for?
Theorem [Valiant] Join and Link can be implemented by vicinal algorithms whp in two steps.
Using Join and Link, one can learn patterns of legth two
of length n > 2?
BA
PJoin(A, B)
Predictive Join [P., Vempala 2015 COLT]
Neurorealistic?
Theorem: PJoin can be implemented by a vicinal algorithm whp in three steps.
Useful? Unsupervised Learning
“Learn a pattern x”
=
“on presentation of x,
create a top-level item I(x), which
will not participate in further PJoins, and
will fire precisely on all subsequent presentations of x”
Vicinal unsupervised learning by PJoin
Algorithm:
While a pattern is presented for time > Tits input neurons fire with probability pnew PJoins are formed with prob q
after a retraction period Rand existing PJoins do their thing
Vicinal unsupervised learning by PJoin
01 0 01
Second presentation
01 0 01
Other patterns: Share and build
00 1 01
Unsupervised Learning
Theorem: Any m patterns in {0, 1}n can be learned whp and with total height O(log m + log n), provided that
T ≥ log n / p, and
D ≥ log n
Simulations
• Patterns with n = 100• all learning activity completed in < 80 steps• sharing as predicted• majority of firing traffic downwards
A tantalizing question
What if the Mind emerges from
Rudimentary primitives
+
Massive hardware
+
“Organic” environments
?
Next Challenges
• Invariants: Clustering-cum-interpolation• Language: A “last-minute adaptation”• Hypothesis: it evolved so as to exploit the
Brain’s strengths• NB: PJoin and “Plink” are ideally fitted for
learning grammar
Sooooo… a prediction
Understanding scientific problems
in terms of algorithmic ideas
will be in the future as widespread
-- and as productive --
as formulating scientific problems
in terms of equations
has been in the past
Thank You!