NICE #8

(The NICE#8 was initially planned to take place in 2020, but had to be postponed to 2021 due to COVID-19.)

Note: the exact order and timing of the talks is so far draft!

Times in the agenda are in CET (Berlin), EDT (New York), PDT (Los Angeles) and UTC. (Some other time zones: Australia, Japan, China, India, ... or only CET ... )

Meeting venue:

• online as zoom video conference (live talks and Q and A). The zoom video conference client software (free of charge, available for Windows, Mac and Linux at zoom.us, also available for iOS and Android in the respective app stores ) is required.
• please register here for NICE 2021

NEUROTECH Forum (15 March 2021)

Info: just before NICE on 15 March 2021 the NEUROTECH Forum II will take place online (free of charge. Registration for the forum event). The topic is "Neuromorphic Computing Technologies: Opportunities, challenges and Applications Roadmap".

Welcome to NICE #8

Organizer Round

Keynote: Lessons from Loihi for the Future of Neuromorphic Computing

The past three years have seen significant progress in neuromorphic computing. The availability of Loihi has enabled a community of over 100 research groups around the world to evaluate a wide range of neuro-inspired algorithms and applications with a neuromorphic chip and toolchain that is sufficiently mature to support meaningful benchmarking. So far these efforts have yielded a number of compelling results, for example in the domains of combinatorial optimization and event-based sensing, control, and learning, while highlighting the opportunities and challenges the field faces for delivering real-world technological value over both the near and long term. This talk surveys the most important results and perspectives we've obtained with Loihi to date.

Why is Neuromorphic Event-based Engineering the future of AI?

The BrainScaleS mobile platform

BrainScaleS is a analog accelerated neuromoprhic hardware architecture. Originally devised to emulate learning in the brain using spike-based models, its research scope has significantly broadened. The most recent addition to the BrainScaleS architecture allows the extension of the analog neuron operation to include rate-based modeling. In this talk we will present the results from a nationwide competition ”Energieeffizientes KI-System”, organized by the German federal ministry of education and research (BMBF) during 2020. The ASIC developed during this competition demonstrated successfully the implementation of an energy-efficient rate-based DCNN using analog verctor-matrix multiplications. To our best knowledge, this is the first time analog computing has been benchmarked in silico by an independent entity with real-world data unknown to the researchers before the evaluation. This talk will present the different technical solutions that made the successful conclusion of the task possible, including software and training aspects.

(break)

Group photo (zoom screenshots)

Evaluating complexity and resilience trade-offs in emerging memory inference machines

Lightning talk: From clean room to machine room: towards accelerated cortical simulations on the BrainScaleS wafer-scale system

The BrainScaleS system follows the principle of so-called "physical modeling", wherein the dynamics of VLSI circuits are designed to emulate the dynamics of their biological archetypes. Neurons and synapses are implemented by analog circuits that operate in continuous time, governed by time constants which arise from the properties of the transistors and capacitors on the microelectronic substrate. This defines our intrinsic hardware acceleration factor of 10000 with respect to biological real-time. The system evolved for over ten years from a lab prototype to a larger installation of several wafer modules. The talk reflects on the development process, the lessons learned and summarize the recent progress in commission- ing and operating the BrainScaleS system. The current state of the endeavor is demonstrated on the example of wafer-scale emulations of functional neural networks.

Closed-loop experiments on the BrainScaleS-2 architecture

The evolution of biological brains has always been contingent on their embodiment within their respective environments, in which survival required appropriate navigation and manipulation skills. Studying such interactions thus represents an important aspect of computational neuroscience and, by extension, a topic of interest for neuromorphic engineering. In the talk, we present three examples of embodiment on the BrainScaleS-2 architecture, in which dynamical timescales of both agents and environment are accelerated by several orders of magnitude with respect to their biological archetypes.

Batch << 1: Why Neuromorphic Computing Architectures Suit Real-Time Workloads

Neuromorphic and AI research at BCAI (Bosch Center for Artificial Intelligence)

Mapping Deep Neural Networks on SpiNNaker2

Open mic / discussion

End of day I

(break)

Tutorials: BrainScaleS and DYNAP-SE

Two tutorials/hands on in parallel:

• BrainScaleS (note: the same BrainScaleS hands on tutorial is also offered on Thursday, 10:00-13:00h CET)
  • about 30 min introduction
  • hands-on usage of the BrainScaleS system (via web browser). Limited number of participants.
• DYNAP-SE (note:the same DYNAP-SE hands on tutorial is also offered on Friday morning)
  • 1-hour live/interactive Dynapse demo: demo on a real Dynapse, take questions and implementing small changes from the audience.
  • 2-hour guided session where participants run a Jupyter notebook with simulations modelling Dynapse. This part is limited to 15 people per session.

For a description please see the tutorials page.

Tutorial: SpiNNaker hands-on

(Note: the same SpiNNaker hands on tutorial is also offered on Thursday, 21:00 - 22:30h CET)

For a description please see the tutorials page.

(break)

NICE - day II

Keynote: From Brains to Silicon -- Applying lessons from neuroscience to machine learning

In this talk we will review some of the latest neuroscience discoveries and suggest how they describe a roadmap to achieving true machine intelligence. We will then describe our progress of applying one neuroscience principle, sparsity, to existing deep learning networks. We show that sparse networks are significantly more resilient and robust than traditional dense networks. With the right hardware substrate, sparsity can also lead to significant performance improvements. On an FPGA platform our sparse convolutional network runs inference 50X faster than the equivalent dense network on a speech dataset. In addition, we show that sparse networks can run efficiently on small power-constrained embedded chips that cannot run equivalent dense networks. We conclude our talk by proposing that neuroscience principles implemented on the right hardware substrate offer the only feasible path to scalable intelligent systems.

A Neuromorphic Future for Classic Computing Tasks

Lightning talk: Benchmarking of Neuromorphic Hardware Systems

With more and more neuromorphic hardware systems for the acceleration of spiking neural networks available in science and industry, there is a demand for platform comparison and performance estimation of such systems. This work describes selected benchmarks implemented in a framework with exactly this target: independent black-box benchmarking and comparison of platforms suitable for the simulation/emulation of spiking neural networks.

Natural density cortical models as benchmarks for universal neuromorphic computers

Throughout evolution, the cortex has increased in volume from mouse to man by three orders of magnitude, while the architecture at the local scale of a cubic millimeter has largely been conserved in terms of the multi-layered structure and the density of synapses. Furthermore, local cortical networks are similar, independent of whether an area processes visual, auditory, or tactile information. This dual universality raises hope that fundamental principles of cortical computation can be discovered. Although a coherent view of these principles still remains missing, the universality motivated researchers already more than a decade go to start to develop neuromorphic computing systems based on the interaction between neurons by delayed point events and basic parameters of cortical architecture.

These systems need to be verified in the sense of accurately representing cortical dynamics and validated in the sense of simulating faster or more energy than software solutions on conventional computers. Such comparisons are only meaningful if they refer to implementations of the same neuronal network model. The role of models changes from mere demonstrations of functionality to quantitative benchmarks. In fields of computer science like computer vision and machine learning the definition of benchmarks helps to quantify progress and drives a constructive competition between research groups. The talk argues that neuromorphic computing needs to advance the development of benchmarks of increasing size and complexity.

A model of the cortical microcircuit [1] exemplifies the recent interplay and co-design of alternative hardware architectures enabled by a common benchmark. The model represents neurons with their natural number of synapses and at the same time captures the natural connection probability between neurons in the local volume. Consequently, all questions on the proper scaling of network parameters become irrelevant. The model constitutes a milestone for neuromorphic hardware systems as larger cortical models are necessarily less densely connected.

As metrics we discuss the energy consumed per synaptic event and the real-time factor. We illustrate the progress in the past few years and show that a single conventional compute node still keeps up with neuromorphic hardware and achieves sub real-time performance. Finally, the talk exposes the limitations of the microcircuit model as a benchmark and positions cortical multi-area models [2] as a biologically meaningful way of upscaling benchmarks to the next problem size.

• [1] Potjans & Diesmann (2014), Cerebral Cortex 24:785–806
• [2] Schmidt, Bakker, Shen, Bezgin, Diesmann, van Albada (2018) PLOS Comput Biol 14(10):e1006359

This work is partially supported by the European Union's Horizon 2020 (H2020) funding framework under grant agreement no. 945539 (Human Brain Project SGA3) and the Helmholtz Association Initiative and Networking Fund under project number SO-092 (Advanced Computing Architectures, ACA).

Poster Lightning Talks

1 min - 1 slide poster appetizers

Poster session A and coffee

Platform-Agnostic Neural Algorithm Composition using Fugu

Lightning talk: Implementing Backpropagation for Learning on Neuromorphic Spiking Hardware

Many contemporary advances in the theory and practice of neural networks are inspired by our understanding of how information is processed by natural neural systems. However, the basis of modern deep neural networks remains the error backpropagation algorithm, which, though founded in rigorous mathematical optimization theory, has not been successfully demonstrated in a neurophysiologically inspired (neuromorphic) circuit. In a recent study, we proposed a neuromorphic architecture for learning that tunes the propagation of information forward and backwards through network layers using a timing mechanism controlled by a synfire-gated synfire chain (SGSC). This architecture was demonstrated in simulation of firing rates in a current-based neuronal network. In this follow-on study, we present a spiking backpropagation algorithm based on this architecture, but including several new mechanisms that enable implementation of the backpropagation algorithm using neuromorphic spiking units. We demonstrate the function of this architecture learning an XOR logic circuit and numerical character recognition with the MNIST dataset on Intel's Loihi neuromorphic chip.

Inductive bias transfer between brains and machines

Machine Learning, in particular computer vision, has made tremendous progress in recent year. On standardized datasets deep networks now frequently achieve close to human or super human performance. However, despite this enormous progress, artificial neural networks still lag behind brains in their ability to generalize to new situations. Given identical training data, differences in generalization are caused by many defining features of a learning algorithm, such as network architecture and learning rule. Their joint effect, called ‘‘inductive bias,’’ determines how well any learning algorithm—or brain—generalizes: robust generalization needs good inductive biases. Artificial networks use rather nonspecific biases and often latch onto patterns that are only informative about the statistics of the training data but may not generalize to different scenarios. Brains, on the other hand, generalize across comparatively drastic changes in the sensory input all the time. I will give an overview on some conceptual ideas and preliminary results how the rapid increase of neuroscientific data could be used to transfer low level inductive biases from the brain to learning machines.

Lightning talk: Spike Latency Reduction generates Efficient Predictive Coding

Latency reduction in postsynaptic spikes is a well-known effect of spiking time-dependent plasticity. We expand this notion for long postsynaptic spike trains on single neurons, showing that, for a fixed input spike train, STDP reduces the number of postsynaptic spikes and concentrates the remaining ones. Then, we study the consequences of this phenomena in terms of coding, finding that this mechanism improves the neural code by increasing the signal-to-noise ratio and lowering the metabolic costs of frequent stimuli. Finally, we illustrate that the reduction in postsynaptic latencies can lead to the emergence of predictions.

Lightning talk: Cognitive Domain Ontologies: HPCs to Ultra Low Power Neuromorphic Platforms

The Cognitively Enhanced Complex Event Processing (CECEP) is an agent-based decision-making architecture. Within this agent, the Cognitive Domain Ontology (CDO) component is the slowest for most applications of the agent. We show that even after acceleration on a high performance server based computing system enhanced with a high end graphics processing unit (GPU), the CDO component does not scale well for real time use on large problem sizes. Thus, to enable real time use of the agent, particularly in power constrained environments (such as autonomous air vehicles), alternative implementations of the agent logic are needed. The objective of this work was to carry out an initial design space search of algorithms and hardware for decision making through the domain knowledge component of CECEP. Several algorithmic and circuit approaches are proposed that span across six hardware options of varying power consumption and weight (ranging from over 1000W to less than 1W). The algorithms range from exact solution producers optimized for running on a cluster of high performance computing systems to approximate solution producers running fast on low power neuromorphic hardware. The loss in accuracy for the approximate approaches is minimal, making them well suited to SWaP constrained systems, such as UAVs. The exact solution approach on an HPC will give confidence that the best answer has been evaluated (although this may take some time to generate).

Open mic / discussion

(break)

Tutorial: BrainScaleS hands-on

(Note: the same BrainScaleS hands on tutorial is also offered on Tuesday evening)

• about 30 min introduction
• hands-on usage of the BrainScaleS system (via web browser). Limited number of participants.

For a description of the pre-requirements, please see the tutorials page.

(break)

NICE - day III

Keynote: Biological inspiration for improving computing and learning in spiking neural networks

The talk will address three new methods:

1. Many type of biological neurons transmit analog values neither through the time of a single spike, nor through their firing rate, but through temporal patterns of a very small number of spikes. If one optimizes spiking neuron models for such communication one arrives at a new and more efficient method for emulating ANNs by SNNs. In particular, this yields the best known performance of SNNs for image classification (on ImageNet) with an average of just two spikes per neuron.

2. Local eligibility traces of synapses that are gated through global learning signals are well known ingredients of synaptic plasticity in brains. We show that these two ingredients enable a principled approximation of BPTT for recurrent SNNs --called e-prop-- that is suitable for on-chip learning on neuromorphic hardware.

3. Brains emit learning signals, for example dopamin, through special brain structures such as VTA, that evolution is likely to have optimized for enabling learning of important new tasks from very few examples. This observation gives rise to a variation of e-prop, called natural e-prop, that enables one-shot and few-shot learning for RSNNs.

For details see:

• C. Stoeckl and W. Maass. Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes. arXiv:2002.00860v4, 2020; in press at Nature Machine Intelligence
• G. Bellec, F. Scherr, A. Subramoney, E. Hajek, D. Salaj, R. Legenstein, and W. Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11:3625, 2020
• F. Scherr, C. Stoeckl, and W. Maass. One-shot learning with spiking neural networks. bioRxiv, 2020.

On the computational power and complexity of Spiking Neural Networks

Evolutionary Optimization for Neuromorphic Systems

Designing and training an appropriate spiking neural network for neuromorphic deployment remains an open challenge in neuromorphic computing. In 2016, we introduced an approach for utilizing evolutionary optimization to address this challenge called Evolutionary Optimization for Neuromorphic Systems (EONS). In this work, we present an improvement to this approach that enables rapid prototyping of new applications of spiking neural networks in neuromorphic systems. We discuss the overall EONS framework and its improvements over the previous implementation. We present several case studies of how EONS can be used, including to train spiking neural networks for classification and control tasks, to train under hardware constraints, to evolve a reservoir for a liquid state machine, and to evolve smaller networks using multi-objective optimization.

An event-based gas sensing device that resolves fast transients in a turbulent environment

Sequence learning, prediction, and generation in networks of spiking neurons

Sequence learning, prediction and generation has been proposed to be the universal computation performed by the neocortex. The Hierarchical Temporal Memory (HTM) algorithm realizes this form of computation. It learns sequences in an unsupervised and continuous manner using local learning rules, permits a context-specific prediction of future sequence elements, and generates mismatch signals in case the predictions are not met. While the HTM algorithm accounts for a number of biological features such as topographic receptive fields, nonlinear dendritic processing, and sparse connectivity, it is based on abstract discrete-time neuron and synapse dynamics, as well as on plasticity mechanisms that can only partly be related to known biological mechanisms.

Here, we devise a continuous-time implementation of the temporal-memory (TM) component of the HTM algorithm, which is based on a recurrent network of spiking neurons with biophysically interpretable variables and parameters. The model learns non-Markovian sequences by means of a structural Hebbian synaptic plasticity mechanism supplemented with a rate-based homeostatic control. In combination with nonlinear dendritic input integration and local inhibitory feedback, this type of plasticity leads to the dynamic self-organization of narrow sequence-specific feedforward subnetworks. These subnetworks provide the substrate for a faithful propagation of sparse, synchronous activity, and, thereby, for a robust, context-specific prediction of future sequence elements as well as for the autonomous replay of previously learned sequences.

By strengthening the link to biology, our implementation facilitates the evaluation of the TM hypothesis based on experimentally accessible quantities. The continuous-time implementation of the TM algorithm permits, in particular, an investigation of the role of sequence timing for sequence learning, prediction and replay. We demonstrate this aspect by studying the effect of the sequence speed on the sequence learning performance and on the speed of autonomous sequence replay.

Poster session b and coffee

Conductance-based dendrites perform reliability-weighted opinion pooling

Cue integration, the combination of different sources of information to reduce uncertainty, is a fundamental computational principle of brain function. Starting from a normative model we show that the dynamics of multi-compartment neurons with conductance-based dendrites naturally implement the required probabilistic computations. The associated error-driven plasticity rule allows neurons to learn the relative reliability of different pathways from data samples, approximating Bayes-optimal observers in multisensory integration tasks. Additionally, the model provides a functional interpretation of neural recordings from multisensory integration experiments and makes specific predictions for membrane potential and conductance dynamics of individual neurons.

Lightning talk: Natural gradient learning for spiking neurons

In many normative theories of synaptic plasticity, weight updates implicitly depend on the chosen parametrization of the weights. This problem relates, for example, to neuronal morphology: synapses which are functionally equivalent in terms of their impact on somatic firing can differ substantially in spine size due to their different positions along the dendritic tree. Classical theories based on Euclidean gradient descent can easily lead to inconsistencies due to such parametrization dependence. The issues are solved in the framework of Riemannian geometry, in which we propose that plasticity instead follows natural gradient descent. Under this hypothesis, we derive a synaptic learning rule for spiking neurons that couples functional efficiency with the explanation of several well-documented biological phenomena such as dendritic democracy, multiplicative scaling and heterosynaptic plasticity. We therefore suggest that in its search for functional synaptic plasticity, evolution might have come up with its own version of natural gradient descent.

Making spiking neurons more succinct with multi-compartment models

Lightning talk: The Computational Capacity of Mem-LRC Reservoirs

Open mic / discussion

(break)

Tutorial: SpiNNaker hands-on

(Note: the same SpiNNaker hands on tutorial is also offered on Wednesday, 10:30-12:00h CET)

For a description please see the tutorials page.

Tutorial:: DYNAP-SE

(Note: the same DYNAP-SE tutorial is also offered on Tuesday evening)

• 1-hour live/interactive Dynapse demo: demo on a real Dynapse, take questions and implementing small changes from the audience.
• 2-hour guided session where participants run a Jupyter notebook with simulations modelling Dynapse. This part is limited to 15 people per session.

(break)

NICE - day IV

Keynote: Bottom-up and top-down neuromorphic processor design: Unveiling roads to embedded cognition

While Moore’s law has driven exponential computing power expectations, its nearing end calls for new roads to embedded cognition. The field of neuromorphic computing aims at a paradigm shift compared to conventional von-Neumann computers, both for the architecture (i.e. memory and processing co-location) and for the data representation (i.e. spike-based event-driven encoding). However, it is unclear which of the bottom-up (neuroscience-driven) or top-down (application-driven) design approaches could unveil the most promising roads to embedded cognition. In order to clarify this question, this talk is divided into two parts.

The first part focuses on the bottom-up approach. From the building-block level to the silicon integration, we design two bottom-up neuromorphic processors: ODIN and MorphIC. We demonstrate with silicon measurement results that hardware-aware neuroscience model design and selection allows reaching record neuron and synapse densities with low-power operation. However, the inherent difficulty for bottom-up designs lies in applying them to real-world problems beyond the scope of neuroscience-oriented applications.

The second part investigates the top-down approach. By starting from the applicative problem of adaptive edge computing, we derive the direct random target projection (DRTP) algorithm for low-cost neural network training and design a top-down DRTP-enabled neuromorphic processor: SPOON. We demonstrate with silicon measurement results that combining event-driven and frame-based processing with weight-transport-free update-unlocked training supports low-cost adaptive edge computing with spike-based sensors. However, defining a suitable target for bio-inspiration in top-down designs is difficult, as it underlies both the efficiency and the relevance of the resulting neuromorphic device.

Therefore, we claim that each of these two design approaches can act as a guide to address the shortcomings of the other.

Lightning talk: Subspace Locally Competitive Algorithms

We introduce subspace locally competitive algorithms (SLCAs), a family of novel network architectures for modeling latent representations of natural signals with group sparse structure. SLCA first layer neurons are derived from locally competitive algorithms, which produce responses and learn representations that are well matched to both the linear and non-linear properties observed in simple cells in layer 4 of primary visual cortex (area V1). SLCA incorporates a second layer of neurons which produce approximately invariant responses to signal variations that are linear in their corresponding subspaces, such as phase shifts, resembling a hallmark characteristic of complex cells in V1. We provide a practical analysis of training parameter settings, explore the features and invariances learned, and finally compare the model to single-layer sparse coding and to independent subspace analysis.

Programming neuromorphic computers: PyNN and beyond

Lightning talk: TBD

BrainScaleS: Development Methodologies and Operating System

The BrainScaleS (BSS) neuromorphic architectures are based on the analog emulation of neuro-synaptic behavior. Neuronal membrane voltages are represented as voltages, model dynamics evolve in a time-continuous manner. Compared to biology the systems run at a typical speedup factor of 1000–10000. This enables the evaluation of effects on long timescales and experiments with many trials. Simultaneously, BSS focuses model configurability and flexibility in plasticity, experiment control and data handling. On BSS-2, this flexibility is facilitated by an embedded SIMD microprocessor located next to the analog neural network core.

The extended configurability, the inclusion of embedded programmability as well as the horizontal scalability of the systems induces additional complexity. Challenges arise in areas such as initial experiment configuration and runtime control, reproducibility and robustness. We present operation and development methodologies implemented for the BSS neuromorphic architectures and walk through the individual components constituting the software stack for BSS platform operation.

Lightning talk: Evolving Spiking Neural Networks for Robot Sensory-motor Decision Tasks of Varying Difficulty

While there is considerable enthusiasm for the potential of spiking neural network (SNN) computing, there remains the fundamental issue of designing the topologies and parameters for these networks. We say the topology IS the algorithm. Here, we describe experiments using evolutionary computation (genetic algorithms, GAs) on a simple robotic sensory-motor decision task using a gene driven topology growth algorithm and letting the GA set all the SNNís parameters. We highlight lessons learned from early experiments where evolution failed to produce designs beyond what we called ìcheap-trickstersî. These were simple topologies implementing decision strategies that could not satisfactorily solve tasks beyond the simplest, but were nonetheless able to outcompete more complex designs in the course of evolution. The solution involved alterations to the fitness function so as to reduce the inherent noise in the assessment of performance, adding gene driven control of the symmetry of the topology, and improving the robot sensors to provide more detailed information about its environment. We show how some subtle variations in the topology and parameters can affect behaviors. We discuss an approach to gradually increasing the complexity of the task that can induce evolution to discover more complex designs. We conjecture that this type of approach will be important as a way to discover cognitive design principles.

Coffee break

Relational Neurogenesis for Lifelong Learning Agents

Lightning talk: Fast and deep neuromorphic learning with first-spike coding

For a biological agent operating under environmental pressure, energy consumption and reaction times are of critical importance. Similarly, engineered systems also strive for short time-to-solution and low energy-to-solution characteristics, but current machine learning solutions struggle to meet especially the latter goal. Back in biology, at the level of neuronal implementation the two goals imply achieving the desired results with as few and as early spikes as possible. In the time-to-first-spike-coding framework, both of these goals are inherently emerging features of learning. We describe a rigorous derivation of learning such first-spike times in networks of leaky integrate-and-fire neurons, relying solely on input and output spike times, and show how it can implement error backpropagation in hierarchical spiking networks. Furthermore, we emulate our framework on the BrainScaleS-2 neuromorphic system and demonstrate its capability of harnessing the chip's speed and energy characteristics to solve the typical machine learning problem of image classification. Finally, we examine how our approach generalizes to other neuromorphic platforms by studying how its performance is affected by typical distortive effects induced by neuromorphic substrates.

Lightning talk: Neuromorphic Computing for Spacecraft’s Terrain Relative Navigation: A Case of Event-Based Crater Classification Task

Beyond Backprop: Different Approaches to Credit Assignment in Neural Nets

Backpropagation algorithm (backprop) has been the workhorse of neural net learning for several decades, and its practical effectiveness is demonstrated by recent successes of deep learning in a wide range of applications. This approach uses chain rule differentiation to compute gradients in state-of-the-art learning algorithms such as stochastic gradient descent (SGD) and its variations. However, backprop has several drawbacks as well, including the vanishing and exploding gradients issue, inability to handle non-differentiable nonlinearities and to parallelize weight-updates across layers, and biological implausibility. These limitations continue to motivate exploration of alternative training algorithms, including several recently proposed auxiliary-variable methods which break the complex nested objective function into local subproblems. However, those techniques are mainly offline (batch), which limits their applicability to extremely large datasets, as well as to online, continual or reinforcement learning. The main contribution of our work is a novel online (stochastic/mini-batch) alternating minimization (AM) approach for training deep neural networks, together with the first theoretical convergence guarantees for AM in stochastic settings and promising empirical results on a variety of architectures and datasets.

Biography: https://sites.google.com/site/irinarish/

Lightning talk: Comparing Neural Accelerators & Neuromorphic Architectures The False Idol of Operations

Accompanying the advanced computing capabilities neural networks are enabling across a suite of application domains, there is a resurgence in interest in understanding what architectures can efficiently enable these advanced computational demands. Both neural accelerators and neuromorphic approaches are emerging at different scales, resource requirements, and enabling capabilities. Beyond the similarity of executing neural network workloads, these two paradigms exhibit significant differences. Accordingly, here we compare neural accelerators and neuromorphic architectures highlighting that operations alone are a lacking singular measure of performance.

Real-time Mapping on a Neuromorphic Processor

Navigation is so crucial for our survival that the brain hosts a dedicated network of neurons to map our surroundings. Place cells, grid cells, border cells, head direction cells and other specialized neurons in the hip- pocampus and the cortex work together in planning and learning maps of the environment [1]. When faced with similar navigation challenges, robots have an equally important need for generating a stable and accurate map. In our ongoing effort to translate the biological network for spatial navigation into a spiking neural network (SNN) that controls mobile robots in real-time, we first focused on simultaneous localization and mapping (SLAM), being one of the critical problems in robotics that relies highly on the accuracy of map representation [2]. Our approach allows us to leverage the asynchronous computing paradigm commonly found across brain areas and therefore has already demonstrated to be a significant energy-efficient solution for 1D SLAM [3], that can spur the emergence of the new neuromorphic processors, such as Intel’s Loihi [4] and IBM’s TrueNorth [5]. In this paper, we expand our previous work by proposing a SNN that forms a cognitive map of an unknown environment and is seamlessly integrated to Loihi.

[1] S. Poulter, T. Hartley, and C. Lever, "The neurobiology of mammalian navigation," Current Biology, vol. 28, no. 17, pp. R1023-R1042, 2018.

[2] G. Grisetti, C. Stachniss, and W. Burgard, "Improved techniques for grid mapping with rao-blackwellized particle filters," IEEE transactions on Robotics, vol. 23, no. 1, p. 34, 2007.

[3] G. Tang, A. Shah, and K. P. Michmizos, "Spiking neural network on neuromorphic hardware for energy- efficient unidimensional SLAM," in IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS), Macau, China, 2019, pp. 1-6.

[4] M. Davies et al., "Loihi: A neuromorphic manycore processor with on-chip learning," IEEE Micro, vol. 38, no. 1, pp. 82-99, 2018.

[5] P. A. Merolla et al., "A million spiking-neuron integrated circuit with a scalable communication network and interface," Science, vol. 345, no. 6197, pp. 668-673, 2014.

Wrap up / farewell

End of NICE 2021