rebooting computing summit 4 : sensible machine...leon chua’s version of the hodgkin-huxley model...
TRANSCRIPT
![Page 1: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/1.jpg)
Rebooting Computing Summit 4 :
Sensible MachineR. Stanley WilliamsSenior FellowHewlett Packard Labs
December 10, 2015
![Page 2: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/2.jpg)
US National Grand Challenge in Future Computing: Sensible Machine
• US BRAIN Initiative - April 22, 2013• OSTP RFI: “Nanotechnology-Inspired Grand Challenges for the Next Decade” – June 17 • Submitted a response to RFI entitled “Sensible Machines” – June 24• Presidential Executive Order: National Strategic Computing Initiative – July 29• OSTP shortlisted ‘Sensible Machines’, asked to ‘develop a program’ – July 30• Worked with IEEE Rebooting Computing and ITRS
– Big thank you to Erik DeBenedictis, Tom Conte, Dave Mountain and many others!• Review of the Chinese Brain-Inspired Computing Research Program – Oct 15• Tom Kalil announces Future Computing Grand Challenge at NSCI workshop – Oct. 20
![Page 3: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/3.jpg)
The “Sensible Machine” response to OSTP RFI
“The central thesis of this white paper is that although our present understanding of brains is limited, we know enough now to design and build circuits that can accelerate certain computational tasks; and as we learn more about how brains communicate and process information, we will be able to harness that understanding to create a new exponential growth path for computing technology.”Our challenge as a community is now to continuously perform more computation per unit energy rather than manufacture more transistors per unit area.
![Page 4: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/4.jpg)
URLs for further information
• White House announcement of Future Computing Grand Challenge:https://www.whitehouse.gov/blog/2015/10/15/nanotechnology-inspired-grand-challenge-future-computing
“Create a new type of computer that can proactively interpret and learn from data, solve unfamiliar problems using what it has learned, and operate with the energy efficiency of the human brain.”nano.gov grand challenges portal:http://www.nano.gov/grandchallenges
• IEEE Rebooting Computing Website:http://rebootingcomputing.ieee.org/archived-articles-and-videos/general/sensible-machine
• Sensible Machine White Paper:http://rebootingcomputing.ieee.org/images/files/pdf/SensibleMachines_v2.5_N_IEEE.pdf
![Page 5: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/5.jpg)
![Page 6: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/6.jpg)
<25 Watts @ 100 Hz
What are the state variables and primativesfor communication and computation?
Ion currents and molecular concentrations:very slow, high energy and inefficient!
How is information processed by a nonlinear dynamical system?
Inspiration from the brain: remarkable power efficiency
![Page 7: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/7.jpg)
Structure of a Neuromorphic Computing Program
7
Example: Chinese Brain-Inspired Computing Research Tsinghua University:
Response to DARPA SyNAPSE and UPSIDE
Operating for three years already
35 faculty from seven departments in eight groupsWell conceived, led and funded (internally by Tsinghua)Already fabbed two chips with a third taped outPresentations at IEDMPlanning to expand program internationally
![Page 8: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/8.jpg)
8
Review of CIBCR, Tsinghua U.
![Page 9: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/9.jpg)
Structure of a US Nanotechnology-Inspired Future Computing Program
1. Connect Theory of Computation with Neuroscience and Nonlinear DynamicsWhat is the computational paradigm? What do spikes really do?Boolean, CNN, Bayesian Inference, Energy-Based Models, Markov Chains
2. Architecture of the Brain and Relation to Computing and LearningTheories of Mind: Albus, Eliasmith, Grossberg, Mead, many others
3. Simulation of Computational Models and Systems
Develop a suite of tools of compact models and detailed analyses4. System Software, Algorithms & Apps – Make it Programmable/Adaptable
At least two thirds of the effort will be in firmware and software
Will this require an open source model?
![Page 10: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/10.jpg)
Structure of a US Nanotechnology-Inspired Future Computing Program
5. Chip Design – System-on-Chip: Accelerators, Learning and ControllersCompatible with standard processors, memory and data bus
6. Chip Processing and Integration – Full Service Back End of Line on CMOS
What facilities are available for general use in the US?DoE Nanoscale Science Research Centers (NSRCs) – e.g. CINTFabbing CMOS in Asia and sending wafers to Europe for BEOL?
7. Devices and Materials – in situ and in operando test and measurementMost likely materials will be adopted from Non-Volatile MemoryAlready more than a decade of experience in commercial grade foundries
One promising path forward utilizes electronic synapses and axons
![Page 11: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/11.jpg)
Processor
Input and Output (I/O)Data and Program Storage/Archive
magnetic disk and tape
Von Neumann Architecture
Von Neumannbottleneck
Memory
![Page 12: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/12.jpg)
Input and Output (I/O)Data and Program Storage/Archive
magnetic disk and tape
Von Neumann Architecture
Von Neumannbottleneck
Combine memory and processing
Processing and memory
![Page 13: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/13.jpg)
Heterogeneous SoCs, all data in memory, programs learned
Von Neumannbottleneck
Input and Output (I/O)
Processing and memory
![Page 14: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/14.jpg)
Combine memory and processing
Input and Output (I/O)
Processing and memory
Eliminate bottleneck, connect to the world
Sensory Input and Output
Neuromorphic Architecture
Photonic Interconnect
![Page 15: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/15.jpg)
15
Neuromorphic special purpose cores for SoCs
What types of special functions or accelerators can we create?
![Page 16: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/16.jpg)
Neural processes can be emulated with memristors
Nonvolatile Memristor- Emerging digital memory/storage- Synapse in neuromorphic circuit
Locally Active (e.g. “Mott”) memristor- Emerging neuronal compute device- Passive “selector” in crossbar memories
Leon Chua, IEEE Trans. Circuit Theory 18, 507 (1971).
![Page 17: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/17.jpg)
Two types of memristors:
Nonvolatile:‘Synaptic’
State stored as resistanceContinuously variableMany Examples
ReRAM – vacancies in oxides
PC RAM – Ge-Sb-TeSTT RAM – spins (binary)
Locally Active:‘Neuronic’ and/or ‘Axonic’
State transmitted as spikeLooks digitalThreshold switching, NDRGain, oscillations, chaos
Mott transitions - oxides
Amorphous Ge-Sb-Te
The semiconductor industry has spent the past decade in developing nonvolatile memories based on these materials and functions
![Page 18: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/18.jpg)
Accelerates many workloads FFT, Metropolis-Hastings, Simulated Annealing
Computing application of nonvolatile memristors
Memristor array = matrix Gij
Computes Matrix-vector dot product VI * G in one time step
Requires non-binary states for each memristor
![Page 19: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/19.jpg)
Sung Hyun Jo, et al. Nano Lett. 10, 1297 (2010)
Viable path toward scalable biomimetic computing?
Neuron (neuristor)
Locally active memristorsCaptures key features of the brain:
1) Non-linear dynamics (“edge of chaos”) of neurons
2) High density architecture, localized memory
i.e. not the von Neuman architecture with physically separated compute and memory !
3) Massive parallelism
Synapse
Nonvolatile memristors
![Page 20: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/20.jpg)
Leon Chua’s Version of the Hodgkin-Huxley Model for neurons
L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International Journal of Bifurcation and Chaos 22 (2012) art. # 1230011.
![Page 21: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/21.jpg)
NbO2 Locally Active “Mott” Memristor – thermoelectric switchingOscillator with DC bias!
rch = 30 nm
![Page 22: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/22.jpg)
A Neuristor inspired by the Hodgkin-Huxley model
M. D. Pickett, et al, Nature Materials 12, 114 (2013).
Implements “All or Nothing” spiking:
500 times faster than a biological neuron
1% of the energy of a neuron
Electronic Action Potential
![Page 23: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/23.jpg)
Grand Challenge addresses a larger community than just nano
–Need to go beyond NNI and involve diverse creative communities– Information technology requires a system-level awareness: Architecture
–Nano devices and circuits will be necessary, but not sufficient for a paradigm shift–Also need insights from neurophysiology (circuits) and psychology (algorithms)
–Revolutionary advances disguised as evolutionary to gain market acceptance–A new nanodevice is useless if it requires a major change (i.e. expense) to a system or
to manufacturing processes (customers don’t pay for performance – they expect it)–Two-thirds (or more) of any computing system today is design and software
–Avoid fads and bandwagons (e.g. Graphene and Deep Learning)–Need a broad investment portfolio of competing technologies and ideas
–Multi-disciplinarity is essential – need deep experts in each domain who can communicate, not lots of broad but shallow neophytes
![Page 24: Rebooting Computing Summit 4 : Sensible Machine...Leon Chua’s Version of the Hodgkin-Huxley Model for neurons L. Chua et al., “Hodgkin-Huxley Axon is made of Memristors,” International](https://reader036.vdocument.in/reader036/viewer/2022071210/60214193054a527a5e177999/html5/thumbnails/24.jpg)
AcknowledgmentsErik P. DeBenedictis, Sandia National LaboratoriesThomas M. Conte, Georgia TechPaolo A. Gargini, ITRSDavid J. Mountain, LPSElie K. Track, nVizix
IEEE Rebooting ComputingITRS
24