We establish a theoretical connection between wavelet transforms and spiking neural networks through scale-space theory. We rely on the scale-covariant guarantees in the leaky integrate-and-fire neurons to implement discrete mother wavelets that approximate continuous wavelets. A reconstruction experiment demonstrates the feasibility of the approach and warrants further analysis to mitigate current approximation errors. Our work suggests a novel spiking signal representation that could enable more energy-efficient signal processing algorithms.

Brains process sensory information to guide behaviour, enabling organisms to adapt to dynamic and unpredictable conditions. Neuromorphic engineering seeks to emulate these neurobiological principles to develop compact, low-power systems capable of real-time sensory-motor integration. This approach addresses some limitations of traditional AI and holds promise for autonomous systems that can interact robustly with the real world. However, most of today’s widely used neuromorphic benchmarks focus primarily on improving accuracy metrics using pre-recorded datasets, often overlooking critical factors such as latency and power consumption. This underscores the need for benchmarks to evaluate real-time performance under noisy, dynamic conditions. To address this need, we developed a system that uses spiking neural networks (SNNs) to control a robotic manipulator in an air-hockey game. In this setup, the automated opponent uses SNNs to process data from an event-based camera, enabling it to track the puck’s movements and respond to the actions of a human player. Our study demonstrates the potential of SNNs to accomplish fast real-time tasks while running on massively parallel hardware. We believe our air-hockey platform provides a versatile testbed for evaluating neuromorphic systems and invites further exploration of advanced algorithms, such as those incorporating trajectory prediction or adaptive learning, which could significantly enhance real-time decision-making and control.

Digital computers have advanced to rival human-level intelligence across creative reasoning and complex problem-solving tasks. Yet, theoretical comparisons with unconventional computational substrates suggest we have only scratched the surface of computational potential. Biological nervous systems have long been studied for their efficiency and robustness, inspiring the invention of neuromorphic hardware. Neuromorphic systems have yet to outperform digital computers, likely due to our limited understanding of the governing computational principles.

This work investigates these computational principles using two modes of inquiry. First, axioms for mixed-signal neural networks are studied by induction as necessary conditions for provably correct neuromorphic computations. Part of the axioms are applied in the Neuromorphic Intermediate Representation, a set of neuromorphic primitives, which is demonstrated to work across more than 12 neuromorphic software and hardware platforms. A second inquiry is made into geometric approaches to event-based vision by deduction. By establishing a direct relationship between neuromorphic primitives and signal transformations, it is demonstrated how neural networks can be imbued with covariance properties that enable them to outperform conventional networks of similar complexity in object tracking tasks. The complementary inductive-deductive approaches provide a more complete lens from which to understand and implement neuromorphic computation.

Additionally, a chapter is dedicated to several openly accessible software projects for evaluating neuromorphic systems on commodity hardware. Apart from being the backbone for the research in this thesis, the accessibility and reproducibility may propagate the research and catalyze community efforts.

Finally, the thesis concludes with a discussion on the broader implications of the above findings and the future trajectory of neuromorphic computation.

The rapid advancements in AI and Machine Learning necessitate a robust computational infrastructure to support cutting-edge research and industrial applications. From the academic and industrial AI community perspective, voiced in the recent ELISE project, the European AI platform is recommended to center around the EuroHPC growing ecosystem. It should be user-driven, easily accessible, powerful, and compliant with European regulations. AI-optimized and dedicated supercomputers for the European AI community are also coming, in addition to upgrading partitions of existing EuroHPC systems to 'AI enabled' stage. Related calls have been initiated in September 2024. Further, conventional EuroHPC systems are suggested to be extended with quantum computing, edge AI, and neuromorphic computing to cater to AI models deployed on network edge devices and sustainability in the long run. The challenges are presented in three case studies, ranging from training Transformers on HPC to LLMs trained federally across three different Euro HPC systems to recent results on hybrid classical-quantum application. This paper concludes with case studies results-informed next steps believed to benefit AI practitioners and the broader AI community.

Biological nervous systems constitute important sources of inspiration towards computers that are faster, cheaper, and more energy efficient. Neuromorphic disciplines view the brain as a coevolved system, simultaneously optimizing the hardware and the algorithms running on it. There are clear efficiency gains when bringing the computations into a physical substrate, but we presently lack theories to guide efficient implementations. Here, we present a principled computational model for neuromorphic systems in terms of spatio-temporal receptive fields, based on affine Gaussian kernels over space and leaky-integrator and leaky integrate-and-fire models over time. Our theory is provably covariant to spatial affine and temporal scaling transformations, and with close similarities to the visual processing in mammalian brains. We use these spatio-temporal receptive fields as a prior in an event-based vision task, and show that this improves the training of spiking networks, which otherwise is known as problematic for event-based vision. This work combines efforts within scale-space theory and computational neuroscience to identify theoretically well-founded ways to process spatio-temporal signals in neuromorphic systems. Our contributions are immediately relevant for signal processing and event-based vision, and can be extended to other processing tasks over space and time, such as memory and control.

Neuroscience education is challenged by rapidly evolving technology and the development of interdisciplinary approaches for brain research. The Human Brain Project (HBP) Education Programme aimed to address the need for interdisciplinary expertise in brain research by equipping a new generation of researchers with skills across neuroscience, medicine, and information technology. Over its ten year duration, the programme engaged over 1,300 experts and attracted more than 5,500 participants from various scientific disciplines in its blended learning curriculum, specialised schools and workshops, and events fostering dialogue among early-career researchers. Key principles of the programme’s approach included fostering interdisciplinarity, adaptability to the evolving research landscape and infrastructure, and a collaborative environment with a focus on empowering early-career researchers. Following the programme’s conclusion, we provide here an analysis and in-depth view across a diverse range of educational formats and events. Our results show that the Education Programme achieved success in its wide geographic reach, the diversity of participants, and the establishment of transversal collaborations. Building on these experiences and achievements, we describe how leveraging digital tools and platforms provides accessible and highly specialised training, which can enhance existing education programmes for the next generation of brain researchers working in decentralised European collaborative spaces. Finally, we present the lessons learnt so that similar initiatives may improve upon our experience and incorporate our suggestions into their own programme.

Spiking neural networks and neuromorphic hardware platforms that simulate neuronal dynamics are getting wide attention and are being applied to many relevant problems using Machine Learning. Despite a well-established mathematical foundation for neural dynamics, there exists numerous software and hardware solutions and stacks whose variability makes it difficult to reproduce findings. Here, we establish a common reference frame for computations in digital neuromorphic systems, titled Neuromorphic Intermediate Representation (NIR). NIR defines a set of computational and composable model primitives as hybrid systems combining continuous-time dynamics and discrete events. By abstracting away assumptions around discretization and hardware constraints, NIR faithfully captures the computational model, while bridging differences between the evaluated implementation and the underlying mathematical formalism. NIR supports an unprecedented number of neuromorphic systems, which we demonstrate by reproducing three spiking neural network models of different complexity across 7 neuromorphic simulators and 4 digital hardware platforms. NIR decouples the development of neuromorphic hardware and software, enabling interoperability between platforms and improving accessibility to multiple neuromorphic technologies. We believe that NIR is a key next step in brain-inspired hardware-software co-evolution, enabling research towards the implementation of energy efficient computational principles of nervous systems. NIR is available at neuroir.org

The value of brain-inspired neuromorphic computers critically depends on our ability to program them for relevant tasks. Currently, neuromorphic hardware often relies on machine learning methods adapted from deep learning. However, neuromorphic computers have potential far beyond deep learning if we can only harness their energy efficiency and full computational power. Neuromorphic programming will necessarily be different from conventional programming, requiring a paradigm shift in how we think about programming. This paper presents a conceptual analysis of programming within the context of neuromorphic computing, challenging conventional paradigms and proposing a framework that aligns more closely with the physical intricacies of these systems. Our analysis revolves around five characteristics that are fundamental to neuromorphic programming and provides a basis for comparison to contemporary programming methods and languages. By studying past approaches, we contribute a framework that advocates for underutilized techniques and calls for richer abstractions to effectively instrument the new hardware class.

In the quest for next-generation sequence modeling architectures, State Space Models (SSMs) have emerged as a potent alternative to transformers, particularly for their computational efficiency and suitability for dynamical systems. This paper investigates the effect of quantization on the S5 model to understand its impact on model performance and to facilitate its deployment to edge and resource-constrained platforms. Using quantization-aware training (QAT) and post-training quantization (PTQ), we systematically evaluate the quantization sensitivity of SSMs across different tasks like dynamical systems modeling, Sequential MNIST (sMNIST) and most of the Long Range Arena (LRA). We present fully quantized S5 models whose test accuracy drops less than 1% on sMNIST and most of the LRA. We find that performance on most tasks degrades significantly for recurrent weights below 8-bit precision, but that other components can be compressed further without significant loss of performance. Our results further show that PTQ only performs well on language-based LRA tasks whereas all others require QAT. Our investigation provides necessary insights for the continued development of efficient and hardware-optimized SSMs.

The Spiking Neural Network Computer Architecture (SpiNNaker) is a massively parallel computing system. As one of the most widespread platforms in the emerging field of neuromorphic engineering, SpiNNaker targets three main areas of research: computational neuroscience, computer science, and robotics. For the latter, the promise of low power computation and the potential for large scale simulations in real-time make SpiNNaker very attractive, especially for autonomous mobile applications. In this context, research groups typically use SpiNNaker's Ethernet interface to inject and extract sensori-motor signals into and from SpiNNaker. However, in cases where the data throughput increases, the on-board Ethernet port constitutes a critical bottleneck. Some groups have overcome such a problem to some extent by developing their own I/O interfaces to connect external devices - - sensors and actuators - - directly to SpiNNaker. However, such custom-developed interfaces allow only limited general applications, and they don't fully exploit the high-speed FPGA-based interconnect offered by the 48-chip SpiNNaker boards.In this manuscript, we present SPIF: a general-purpose FPGA-based SpiNNaker Peripheral Interface board that overcomes SpiNNaker's communication bottleneck by connecting to its native High-Speed Serial Links (HSSLs). We evaluate SPIF's performance in terms of event throughput and latency. Finally, we demonstrate SPIF's capabilities by feeding events from a high-resolution event camera into a real-time spiking convolutional neural network. The system can track the position of a small and extremely fast but salient stimulus in the visual field with negligibly low latency.