PGI-15 Tutorial
Introduction to Neuromorphic Computing
02.11.2023 | Emre Neftci, Susanne Kunkel, Jamie Lohoff, and Willem Wybo
Plan of PGI-15 Tutorial
- Today and Nov. 16: Introduction to Neuromorphic Computing [45 min]
- History of neuromorphic engineering
- Neuromorphic engineering today
- Neuroscience top-down modeling and neuromorphic computing: ML, ANN and SNN
- Synaptic plasticity and learning in the brain and hardware challenges
- Machine learning inspired models of neural computation (“Neuroai”)
- Nov 23. Susanne Kunkel Simulation of large spiking neural networks
- Nov 30. Jamie Lohoff A Primer on Deep Reinforcement Learning
- Dec 7. Willem Wybo Dendritic Models of Neural Computation
History of Artificial Intelligence and Neural Networks
McCulloch and Pitts
Slides Chrstian Gamrat, ESSDERC 2012
Perceptron Learning Rule
Slides Chrstian Gamrat, ESSDERC 2012
Limits of Perceptrons
Slides Chrstian Gamrat, ESSDERC 2012
History of Artificial Intelligence and Neural Networks
Modern AI/ML
Much AI/ML progress in the last decade can be attributed to better hardware and more data
Good old online backpropagation for plain multilayer perceptrons yields a very low 0.35% error rate on the MNIST handwritten digits benchmark. All we need to achieve this best result so far are many hidden layers, many neurons per layer, numerous deformed training images to avoid overfitting, and graphics cards to greatly speed up learning.
Ciresan et al. 2010
Processor Clock Speeds are Stalling
Why the Brain can do things a Computer Can't
Biological Neuron Models
See INM-6 Tutorial
The Distributed Organization of the Cortex
Distributed Organization of the Primate Visual Cortex
Felleman and Van Essen, Cerebral cortex, 1991
Pyramidal cells (Neurons) in the Cortex
Ramon y Cajal, 1911
See INM-6 TutorialTheoretical insights: Hopfield Nets
Slides Chrstian Gamrat, ESSDERC 2012
Meanwhile at Caltech
Slide by Tobi Delbruck, 2007
The Physics of Computation
Slide by Tobi Delbruck, 2007
The Physics of Computation
|
The constraints on our analog silicon systems are similar to those on neural systems: wire is limited, power is precious, robustness and reliability are essential.
Mead, 1989
|
|
|
The constraints on our analog silicon systems are similar to those on neural systems: wire is limited, power is precious, robustness and reliability are essential.
Mead, 1989
Analysis by Synthesis
Douglas and Mahowald
Emulation of the bio-physics of neural systems circuits
Chicca, Stefanini, and Indiveri, 2013
Event-based Communication
Deiss et al. 1993
The HydroNeuron
Neuromorphic Systems Development
Event-based Communication
-
Rich analog dynamics and sparsity (only spike events routed).
Local computation (Non von Neumann): + Scalability - Algorithmic challenge
Research in Neuromorphic Computing
There is a wide gap between AI and Machine Learning
