cognitive computing via synaptronics and supercomputing
DESCRIPTION
Cognitive Computing via Synaptronics and Supercomputing. - PowerPoint PPT PresentationTRANSCRIPT
© 2008 IBM Corporation
Cognitive Computing via Synaptronics and Supercomputing
© 2008 IBM Corporation
"The information that comes from deep in the evolutionary past we call genetics. The information passed along from hundreds of years ago we call culture. The information passed along from decades ago we call family, and the information offered months ago we call education. But it is all information that flows through us. The brain is adapted to the river of knowledge and exists only as a creature in that river. Our thoughts are profoundly molded by this long historic flow, and none of us exists, self-made, in isolation from it."
© 2008 IBM Corporation
Inflection Point 1: Neuroscience has matured
1414 pages
© 2008 IBM Corporation
Inflection Point 2: Supercomputing meets Brain
© 2008 IBM Corporation
Memory Computation
Communication
Mammalian-scale simulation in near real-time?
© 2008 IBM Corporation
LLNL Dawn
BG/P
May, 2009
Human
22 x 109
220 x 1012
Rat
56 x 106
448 x 109
Mouse
16 x 106
128 x 109
N:
S:
Monkey
2 x 109
20 x 1012
Cat
763 x 106
6.1 x 1012
Almaden
BG/L
December, 2006
Watson
BG/L
April, 2007
WatsonShaheen
BG/P
March, 2009
BlueGene Meets Brain
Latest simulations achieve unprecedented scale of
109 neurons and 1013 synapses
New results for SC09
© 2008 IBM Corporation
Inflection Point 3: Nanotechnology meets Brain
© 2008 IBM Corporation
Rat Human
Power 50 mW 20 W
Space 6 cm2 2,400 cm2
Brain Neuromorphic Electronics
~1010 synapses/cm2 1010 intersection/cm2 in 100 nm crossbar
~106 Neurons/cm2 ~5x108 transistors/cm2 in state of the art CMOS
~5 x 108 long range axons @ ~1 Hz
~30 Gbit/sec multiplexed digital addressing
Novel non-von Neumann Architectures are necessary
Brain can be realized in modern electronics
Data from Todd Hylton
© 2008 IBM Corporation
Turning Back the Clock
Digital, synchronous conventional, 5GHz(compare Power 6, 2008)
Digital, asynchronous, 100 kHz(compare ENIAC, 1946)
Digital, semi-synchronous, 5 MHz(compare IBM PC/8088, 1978)
Digital-Analog, asynchronous, clockless(compare the Brain)
Commandment:Do what is necessary, when it is necessary, and only that which is necessary.
© 2008 IBM Corporation
Network Architecture of the White Matter Pathways in the Macaque Brain
PNAS (July 2010)
Dharmendra S ModhaIBM Research – Almaden
Raghavendra Singh IBM Research – India
© 2008 IBM Corporation
The connection model Cortex has evolved such that it is
organized into areas with distinct structural and functional properties
• Primary sensory areas• Association areas• Motor areas
The white matter (myelinated nerve cell) underneath the outer covering of gray matter (nerve cell bodies), interconnects different regions of the central nervous system and carries nerve impulses between neuron
Model each area as a node and each connection as an edge in a graph
– Analysis and Visualization of the brain• Wire length minimization• Organizational model that suggest the flow
of information from input of sensory signals to the eventual output by motor neurons
– Use model to simulate dynamics in the simulator
© 2008 IBM Corporation
CoCoMac: Connectivity data on the Macaque brain
413 literature reports
7007 brain sites
8003 mapping details
2508 tracer injections
39748 connection details
Rolf Kotter, Klass Stephen, 2000
© 2008 IBM Corporation
FV91-V1
AP84-TE
PG91b-IT
BP82-46
SP89a-46
FV91-V4FV91-TF
RV99-TF
RV99-CA1
BR98-CA1
FV91-TH
BR98-TH
=
כ כ
PFCd 9=
PCi IPL PF#1 7b
= =
CCa 24 24d 24c CMAr
24c
ככ
= = = =
=
Divergent Nomenclature and Abundant Conflicts
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM CorporationBundling Algorithm by Holten, 2006
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM Corporation
© 2008 IBM CorporationKaiser, Hilgetag, 2006
© 2008 IBM Corporation
Species Study Areas Connections
Monkey Felleman, Van Essen, 91 32 305
Young, 93 70 700
Kaiser, Hilgetag, 06 95 2,402
Cat Scannell et al., 95 65 1,139
Scannell et al., 99 95 1,500
Notable Collations
© 2008 IBM Corporation
Species Study Areas Connections
Monkey Felleman, Van Essen, 91 32 305
Young, 93 70 700
Kaiser, Hilgetag, 06 95 2,402
Cat Scannell et al., 95 65 1,139
Scannell et al., 99 95 1,500
Monkey This Collection 383 6,602
Notable Collations
© 2008 IBM Corporation
© 2008 IBM CorporationCingulum Bundle
© 2008 IBM CorporationUncinate Fasciculus
© 2008 IBM Corporation
Complete Cortex, Thalamus, Basal Ganglia
Comprehensive Includes every study in CoCoMac
Consistent Every connection can be tracked back
Concise 6,877 areas to 383
Coherent Unified hierarchical parcellation
Colossal 3 times larger than previous network
C, C, C, C, C, and C
wetware to software
© 2008 IBM Corporation
Nodes 383
Edges 6,602
Density 4.5% of possible connections exist
Reciprocity 42%
SCC 351 areas, 6,491 edges
Aggregate Statistics
© 2008 IBM Corporation
Brain is small-world
Brain Random(100 trials)
Diameter 6 3.93
Characteristic Path Lengh 2.62 2.30
Average Clustering Coefficient 0.33 0.0528
Reciprocity 42% 5.34%
SCC: 351 areas, 6,491 connections
© 2008 IBM Corporation
“Organized Complexity” – Weaver, 1948
© 2008 IBM Corporation
Degree Distribution Consistent with Exponential
© 2008 IBM Corporation
Prefrontal Cortex is Topologically Central
© 2008 IBM Corporation
© 2008 IBM Corporation
Brain is small-world, Core is “tiny”-world!
Brain Core
Diameter 6 4
Characteristic Path Lengh 2.62 1.95
Average Clustering Coefficient 0.33 0.39
Core contains only 32% of vertices yet 88% of all edges originate or terminate in the core
© 2008 IBM Corporation
Core contains correlated-anti-correlated networksand may be a key to consciousness
Fox, Snyder, Vincent, Corbetta, Van Essen, and Raichle, 2005
© 2008 IBM Corporation
Inter-chip Connectivity
© 2008 IBM Corporation
Rent’s Rule Rent's rule pertains to the organization of computing logic, specifically the
relationship between the number of external signal connections (C) to a logic block with the number of logic gates (N) in the logic block
E.F. Rent observed a power-law relationship in the 1960’s - the law has been shown to hold true for small circuits upto mainframe computers
€
C = kN p
Intrinsically it’s a surface area (wire) to volume (number of nodes) relationship– Represents a cost-efficient solution to the challenge of embedding a high dimensional functional
interconnect topology in a relatively low dimensional physical space with economical wiring costs
Circuits with greater logical capacity have higher values of Rent parameter– Microprocessor (0.45), Gates Arrays (0.5), High speed Computers (0.63)
For 2D layouts p> 0.5 implies that wires must grow longer as circuit size increases; global connections dominate over local connections for large p– The relative contribution of wiring to layout area will grow with the size of circuit to allow space for a greater
number of wires to pass between adjacent nodes, increasing the node-to-node spacing
Allometric scaling
– Gray (physical) to white (logical) matter scaling - Zhang & Sejnowski
0 ≤ p≤1 is the Rent parameter and k is the Rent coefficient.
© 2008 IBM Corporation
Rent’s Rule
High value of p
– Topological dimensionality of network greater than 3, i.e., greater than the dimensionality of the Euclidean space in which the network is embedded
– Communication is a significant factor of power and space
– Tradeoff between wiring costs and greater logical capacity.• Rewiring the network
so as to reduce its topological dimension results in loss of functional modularity
© 2008 IBM Corporation
© 2008 IBM Corporation
Nicolaus Steno, 1669
“white matter is nature’s finest masterpiece”
© 2008 IBM Corporation
Owing both to limitations in hardware and architecture, these (convential) machines are of limited utility in complex, real-world environments, which demand an intelligence that has not yet been captured in an algorithmic-computational paradigm. As compared to biological systems for example, today’s programmable machines are less efficient by a factor of one million to one billion in complex, real-world environments. The SyNAPSE program seeks to break the programmable machine paradigm and define a new path forward for creating useful, intelligent machines.
The vision for the anticipated DARPA SyNAPSE program is the enabling of electronic neuromorphic machine technology that is scalable to biological levels. Programmable machines are limited not only by their computational capacity, but also by an architecture requiring (human-derived) algorithms to both describe and process information from their environment. In contrast, biological neural systems (e.g., brains) autonomously process information in complex environments by automatically learning relevant and probabilistically stable features and associations. Since real world systems are always many body problems with infinite combinatorial complexity, neuromorphic electronic machines would be preferable in a host of applications—but useful and practical implementations do not yet exist.