("Theory day")

Agenda as .pdf download

The agenda as of 11 April 2023 can be downloaded here as .pdf.

Venue

UTSA Student Union, H-E-B University Center, 1 UTSA Circle, San Antonio, TX 78249, Texas, United States of America.

Also available: a schematic view as .pdf

Registration

Please follow the link on the registration page to register for the workshop.

• Dr. Dhireesha Kudithipudi, UT San Antonio
• Dr. Brad Aimone, Sandia National Laboratories
• Dr. Johannes Schemmel, Kirchhoff-Institute for Physics, Heidelberg University
• Dr. Suma George Cardwell, National Laboratories

Neuroevolution, or design of neural networks through evolutionary algorithms, has long been used to solve tasks where gradients are not available, such as partially observable decision tasks. Recently it has also turned out useful in designing complex deep learning architectures. I will outline how the approach can result in complex architectures beyond human designs, complex behavior beyond human expectations, and solutions that combine human expertise and evolutionary discovery synergetically, with examples in vision, language, robotics, game playing, and decision making.

Authors: Jens Egholm Pedersen and Jörg Conradt.

We present a novel method to efficiently process event streams on conventional hardware, along with a freely available implementation: AEStream. AEStream provides at least 2x throughput compared to conventional parallelization mechanisms and at least 5x faster memory management on GPUs. Our method operates directly on event-address representations, allowing us to (1) freely combine input-output pairs and (2) directly interface event-based peripherals, such as neuromorphic hardware and event cameras. github.com/norse/aestream

Authors: Bradley Theilman and James B. Aimone

(Unfortunately no slides were in the talk video capture - therefore there is no video of the talk available here)

Authros: Denis Kleyko, Christopher Kymn, Bruno A. Olshausen, Friedrich T. Sommer and E. Paxon Frady.

(Unfortunately no slides were in the talk video capture - therefore there is no video of the talk available here)

This presentation will introduce a way of computing higher-order features, which have been recently proposed for the use within the reservoir computing, via compositional distributed representations formed by the framework of hyperdimensional computing. At the implementational level, the proposed mechanism can be realized as a network of Sigma-Pi neurons.

(this talk video has a gap)

We present major design issues for large-scale neuromorphic computing systems, and some of the trade-offs in designing hardware and software for such systems. Many of the detailed hardware trade-offs that have significant impact on overall energy efficiency depend strongly on the networks being mapped to the hardware. We describe ongoing work on creating a quantitative, full-stack approach to evaluating the trade-offs in neuromorphic system design, enabled by recently developed open-source tools for the design and implementation of asynchronous digital systems.

Authors: Claire Plunkett and Frances Chance.

Coordinate transformations are a fundamental operation that must be performed by any animal relying upon sensory information to interact with the external world. We present a neural network model that performs a coordinate transformation from the dragonfly eye's frame of reference to the body's frame of reference while hunting. We demonstrate that the model successfully calculates turns required for interception, and discuss how future work will compare our model with biological dragonfly neural circuitry and guide neural-inspired neuromorphic implementations of coordinate transformations.

Authros: Adam McCaughan, Cory Merkel, Bakhrom Oripov, Andrew Dienstfrey, Sae Woo Nam and Sonia Buckley.

Authors: Anurag Daram and Dhireesha Kudithipudi.

Continual learning is challenging for deep neural networks, mainly because of catastrophic forgetting, the tendency for accuracy on previously trained tasks to drop when new tasks are learned. Although several biologically-inspired techniques have been proposed for mitigating catastrophic forgetting, they typically require additional memory and/or computational overhead. Here, we propose a novel regularization approach that combines neuronal activation-based importance measurement with neuron state-dependent learning mechanisms to alleviate catastrophic forgetting in both task-aware and task-agnostic scenarios. We introduce a neuronal state-dependent mechanism driven by neuronal activity traces and selective learning rules, with storage requirements for regularization parameters that grow asymptotically slower with network size - compared to schemes that calculate weight importance, whose storage grows quadratically. The proposed model, NEO, is able to achieve performance comparable to other state-of-the-art regularization based approaches to catastrophic forgetting, while operating with a reduced memory overhead.

Authors: Karan Patel and Catherine Schuman.

Shuttle leaves at 18:00h from the meeting place and goes to "UTSA, San Pedro 1" (place of the welcome reception)

Address of the place: 506 Dolorosa St, San Antonio, TX 78204

(For people using their own car: parking space should likely be available at "Dolorosa Lot")

Shuttle leaves at 21:00h = 9:00 pm and returns to "UTSA main campus" (conference venue)

("Hardware" day)

We present neuromorphic cognitive computing systems-on-chip implemented in custom silicon compute-in-memory neural and memristive synaptic crossbar array architectures that combine the efficiency of local interconnects with flexibility and sparsity in global interconnects, and that realize a wide class of deeply layered and recurrent neural network topologies with embedded local plasticity for on-line learning, at a fraction of the computational and energy cost of implementation on CPU and GPGPU platforms. Co-optimization across the abstraction layers of hardware and algorithms leverage inherent stochasticity in the physics of synaptic memory devices and neural interface circuits with plasticity in reconfigurable massively parallel architecture towards high system-level accuracy, resilience, and efficiency for natural intelligence at the edge. Adiabatic energy recycling in charge-mode crossbar arrays permit extreme scaling in energy efficiency, approaching that of synaptic transmission in the mammalian brain.

Deep learning gained an insane amount of traction due to the availability of miniaturized sensors, data availability, and the broad open-source culture. The chip design flow from RTL to bring-up is amongst the most closed and secretive processes known to man. This means that there is a very high barrier to designing and building integrated circuits. This presentation explores how neuromorphic computing is the perfect battle ground for harnessing recently open-sourced design flows, process design kits, and memory compilers. I will talk about several neuromorphic chips we have taped out in the SkyWater process, including OpenSpike, and how these can promote reproducibility of both software and silicon. After all, open sourcing is what enabled the deep learning hype train. The next station is silicon.

Authors: Ali Safa, Jonah Van Assche, Charlotte Frenkel, André Bourdoux, Francky Catthoor and Georges Gielen.

Authors: James Knight and Thomas Nowotny.

Intuitive and easy to use application programming interfaces such as Keras have contributed majorly to the rapid acceleration of machine learning with artificial neural networks. Building on our recent works on translating ANNs to SNNs and training classifiers with eProp, we here present the mlGeNN interface as an easy way to define, train and test spiking neural networks on our efficient GPU based GeNN framework. We illustrate the use of mlGeNN by investigating the performance of a number of shallow spiking neural networks trained with the e-prop learning rule to recognise hand gestures from the DVS gesture dataset. We find that not only is mlGeNN vastly more convenient to use than the lower level PyGeNN interface, the new freedom to effortlessly and rapidly prototype different network architectures also gave us an unprecedented overview over how e-prop compares to other recently published results on the DVS gesture dataset across architectural details.

Brain’s functionality has long been postulated to be related to its structure. This structure-function relationship is present at different spatial and hierarchical scales, going from neurons and synapses, to dendritic arbors and of course to its connectome. Inspired by this, I will present neuromorphic circuits and architectures, incorporating resistive memory into different levels of this computational hierarchy. The resistive memory can be used as the knob that changes the form, and thus function, of the synapses, neurons, dendritic arbors and connectivity for hierarchical sensory signal processing and on-chip learning.

Author: Davin Browner.

Hardware design for application of online machine learning is complicated by a number of facets of conventional ANN frameworks, e.g. deep neural networks (DNNs), such as reliance on non-temporally local offline learning, potential difficulties in transfer from model to substrates, and issues with processing of noisy sensory data using energy-efficient and asynchronous information processing modalities. Analog or mixed-signal spiking neural networks (SNNs) have promise for lower power, temporally localised, and stimuli selective online sensing and inference but are difficult to design and fabricate at low cost. Investigation of beyond-CMOS alternative substrates including organic and organometallic compounds may be worthwhile for development of unconventional neuromorphic hardware with pseudo-spiking dynamics. Here, polymeric organometallic ferroelectric diodes (POMFeDs) are introduced as a hardware platform for development of printable ferroelectric in-sensor SNNs.

Authors: Philipp Spilger, Elias Arnold, Luca Blessing, Christian Mauch, Christian Pehle, Eric Müller and Johannes Schemmel.

Neuromorphic systems require user-friendly software to support the design and optimization of experiments. In this work, we address this need by presenting our development of a machine learning-based modeling framework for the BrainScaleS-2 neuromorphic system. This work represents an improvement over previous efforts, which either focused on the matrix-multiplication mode of BrainScaleS-2 or lacked full automation. Our framework, called hxtorch.snn, enables the hardware-in-the-loop training of spiking neural networks within PyTorch, including support for auto differentiation in a fully-automated hardware experiment workflow. In addition, hxtorch.snn facilitates seamless transitions between emulating on hardware and simulating in software. We demonstrate the capabilities of hxtorch.snn on a classification task using the Yin-Yang dataset employing a gradient-based approach with surrogate gradients and densely sampled membrane observations from the BrainScaleS-2 hardware system.

Authors: Ali Safa, Ilja Ocket, Francky Catthoor and Georges Gielen.

In this talk, we introduce SupportHDC, a novel HDC design framework that can jointly optimize system accuracy and sparsity in an automated manner, in order to trade off classification performance and hardware implementation overheads. We illustrate the inner working of the framework on two bio-signal classification tasks: cancer detection and arrhythmia detection. We show how SupportHDC enables the system designer to choose the final design solution from the accuracy-sparsity trade-off curve produced by the framework. The python code for reproducing our experiments is released as open-source with the hope of being beneficial to future research.

Artificial Intelligence, or AI, has become pervasive in a wide variety of domains, from image and video classification to speech recognition, translation and text generation, just to cite a few examples. These models come with accuracies which improve year after year, however at the cost of huge training and inference computational effort both in terms of energy and time. For this reason, research groups from both academia and industry are developing novel approaches to accelerate computation. We have recently developed an in-memory analog computing chip with more than 35 million Phase-Change Memory devices, analog peripheral circuitry and massive parallel routing to accelerate communication between inputs, outputs and analog cores. We demonstrate that analog computing shows significant advantages in terms of power and speed, yet retaining high accuracy on neural networks taken from both image classification and language processing tasks.

Authors: Wesley Brigner, Naimul Hassan, Xuan Hu, Christopher Bennett, Felipe Garcia-Sanchez, Can Cui, Alvaro Velasquez, Matthew Marinella, Jean Anne Incorvia and Joseph S. Friedman.

Authors: Frances Chance and Suma Cardwell.

Shunting inhibition is a potential mechanism by which biological systems multiply two time-varying signals, most recently demonstrated in single neurons in the fly visual system. Our work demonstrates this effect in a biological neuron model and also models the equivalent circuit in neuromorphic hardware. Here we demonstrate how this mechanism can be leveraged in neuromorphic dendrites.

Location: iH-E-B Ballroom 1.104

("Applications day")

Authors: Sebastian Siegel, Tobias Ziegler, Younes Bouhadjar, Tom Tetzlaff, Rainer Waser, Regina Dittmann and Dirk Wouter.

We present measurement results on a memristive / 130nm CMOS co-integrated chip of high-order sequence learning experiments with the MemSpikingTM algorithm which was developed as a hardware friendly version of SpikingTM, a biologically plausible version of the Hierarchical Temporal Memory's (HTM) Temporal Memory.

• Joe Hays (U.S Naval research Lab)
• Andrey Kanaev (NSF)
• Tina Kaarsberg (DOE)
• Jano Costard (SPRIN-D, Germany)
• Clare Thiem (AFRL)

Authors: Kenneth Stewart, Timothy Shea, Noah Pacik-Nelson, Eric Gallo and Andreea Danielescu.

(unfrotunately no talk video captured)

Despite the maturity and availability of speech recognition systems, there are few available spiking speech recognition tasks that can be implemented with current neuromorphic systems. The methods used previously to generate spiking speech data are not capable of encoding speech in real-time or encoding very large modern speech datasets efficiently for input to neuromorphic processors. The ability to efficiently encode audio data to spikes will enable a wider variety of spiking audio datasets to be available and can also enable algorithmic development of real-time neuromorphic automatic speech recognition systems. Therefore, we developed speech2spikes, a simple and efficient audio processing pipeline that encodes recorded audio into spikes and is suitable for real-time operation with low-power neuromorphic processors. To demonstrate the efficacy of our method for audio to spike encoding we show that a small feed-forward spiking neural network trained on data generated with the pipeline achieves 88.5% accuracy on the Google Speech Commands recognition task, exceeding the state-of-the art set by Spiking Speech Commands, a prior spiking encoding of the Google Speech Commands dataset, by over 10%. We also demonstrate a proof-of-concept real-time neuromorphic automatic speech recognition system using audio encoded with speech2spikes streamed to an Intel Loihi neuromorphic research processor.

Authors: Diego Chavez Arana, Alpha Renner and Andrew Sornborger.

(unfrotunately no talk video captured)

Authors: Charles Rizzo, Catherine Schuman and James Plank.

(unfrotunately no talk video captured)

In this work, we address the problem of training a neuromorphic agent to work on data from event-based cameras. Although event-based camera data is much sparser than standard video frames, the sheer number of events can make the observation space too complex to effectively train an agent. We construct multiple neuromorphic networks that downsample the camera data so as to make training more effective. We then perform a case study of training an agent to play the Atari Pong game by converting each frame to events and downsampling them. The final network combines both the downsampling and the agent. We discuss some practical considerations as well.

Authors: George Brayshaw, Martin Pearson and Benjamin Ward-Cherrier.

(unfrotunately no talk video captured)

Authors: Madeleine Abernot, Sylvain Gauthier, Théophile Gonos and Aida Todri-Sanial.

Authors: Jens Egholm Pedersen, Raghav Singhal and Jörg Conradt.

We propose a new method to accurately predict spatial coordinates of objects from event data using spiking neurons without temporal averaging. Our method achieves accuracies comparable to artificial neural networks, demonstrates faster convergence, and is directly portable to neuromorphic hardware. In this talk, we will present our model, along with unpublished experimental data, and discuss its generalization to real-life settings. github.com/jegp/coordinate-regression/

Authors: Wallace Lawson, Anthony Harrison and Greg Trafton.

Our autonomous robot, Bight, can be a reliable teammate that is capable of assisting in performing routine maintenance tasks on a Naval vessel. In this paper, we consider the task of maintaining the electrical panel. A vital first step is putting the robot into the correct position to view all of the parts of the electrical panel. The robot can get close, but the arm of the robot will need to move to where it can see everything. Here, we propose to solve this using a sigma delta spiking network that is trained using deep Q learning. Our approach is able to successfully solve this problem at varying distances. While we show how this works on this specific problem, we believe this approach to be general enough to be applied to any similar problem.

Likely three slots in parallel

Confirmed tutorials:

• An Introduction to a Simulator for Super Conducting Optoelectronic Networks (Sim-SOENs)
• Sandia – Fugu Introductory Tutorial (offered twice with the same content)
• N2A -- An IDE for neural modeling
• Hands-on BrainScaleS - analog accelerated neuromorphic compute hardware (The hands-on session is offered twice with the same content). The BrainScaleS hardware systems are available for use online
• Intel Loihi 2: Build more impactful neuromorphic applications with Intel Loihi 2 and the open-source Lava framework

An Introduction to a Simulator for Super Conducting Optoelectronic Networks (Sim-SOENs)

This tutorial will suffice to impart a functional understanding of Sim-SOENs. Starting with the computational building blocks of SOEN neurons, we will cover the nuances and processing power of single dendrites, before building up to dendritic arbors within complex neuron structures. We will find it is straightforward to implement arbitrary neuron structures and even dendritic-based logic operations. Even at this single neuron level, we will already demonstrate efficacy on basic computational tasks. From there we will scale to network simulations of many-neuron systems, again with demonstrative use-cases. By the end of the tutorial, participants should be able to easily generate custom SOEN neuron structures and networks. These lessons will apply directly to researching in the computational paradigm that is to be instantiating on the burgeoning hardware of SOENs.
Format: Examples and instructions will be given in the form of Jupyter Notebook tutorials (already well into development). If it is conducive to the conference environment, these notebooks may be available for download and use in real-time. If this latter format is the case, practice exercises can be derived for active learning.

N2A -- An IDE for neural modeling

N2A is a tool for editing and simulating large-scale/complex neural models. These are written in a simple equation language with object-oriented features that support component creation and reuse. The tool compiles these models for various hardware targets ranging from neuromophic devices to supercomputers.
Format: The first hour will provide a general introduction to the integrated development environment (IDE) and cover basic use cases: model editing, running a simulation, sharing models via Git, and running parameter sweeps. The second hour will cover the basic LIF class hierarchy, techniques for designing your own component set, and integration with Sandia's Fugu tool.
Special Requirements: This will be a hands-on tutorial. N2A may be downloaded from https://github.com/frothga/n2a and run on your personal laptop.

BrainScaleS

A hands-on tutorial for online interactive use of the BrainScaleS neuromorphic compute system: from the first log-in via the EBRAINS Collaboratory to interactive emulation of small spiking neural networks. This hands-on tutorial is especially suitable for beginners (more advanced attendants are welcome as well). We are going to use the BrainScaleS tutorial notebooks for this event.
For using the BrainScaleS system during the tutorial (and also independently of the tutorial for own research, free of charge for evaluation) an EBRAINS account (also free of charge) is needed (get an EBRAINS account here).
More info on how to get started using BrainScaleS.
Format: Introductory presentation, followed by interactive hands-on tutorials. The attendants of the tutorial can a webbrowser on their own laptops to execute and change provided tutorials and explore on their own. Attendants will be able to continue accessing the systems with a generous test-quota also after the event
Preparation: Best to get an EBRAINS account here) ahead of time. We can also create a guest-account on the spot.

Fugu Introductory Tutorial

The tutorial will cover the basic design and practice of Fugu, a software package for composing spiking neural algorithms. We will begin will an introductory presentation on the motivation, design, and limitations of Fugu. Then, we will do two deep dive, interactive tutorials using jupyter notebooks. The first will cover how to use Fugu with pre-existing components, we call Bricks. The second will cover how to build a custom brick to perform a particular algorithm. In this case, the algorithm we choose will be an 80-20 network.
Format: Interactive
Preparation: Please clone and install: https://github.com/snl-nerl/fugu

Intel Loihi 2: Build more impactful neuromorphic applications with Intel Loihi 2 and the open-source Lava framework

Tim Shea from Intel Labs will demonstrate how you can program applications using the open-source Lava framework for neuromorphic computing and how to compile and run those applications on Intel Loihi 2 hardware. Lava is an excellent platform for neuromorphic researchers seeking more real-world impact because the high-level, modular API makes it easy for other labs to replicate your work while the flexible compiler architecture makes it easy to distribute your models across conventional and neuromorphic hardware. In this tutorial, you will learn how to build and run several example applications in Lava, including a deep learning model, a Dynamic Neural Field algorithm, a mathematical optimizer.

Format: This tutorial will introduce application programming in Lava through a series of Jupyter notebook tutorials. Attendees can follow along building the applications on their own laptops or using any free cloud-based notebook (e.g. Google Colab). Each application can be run locally on a standard CPU and the presenter will demonstrate how to run the examples on an Intel Kapoho Point neuromorphic system. All the necessary code and instructions are available at github.com/lava-nc.