Direct training high-performance deep spiking neural networks: a review of theories and methods

Frontiers in Neuroscience, Journal Year: 2024, Volume and Issue: 18

Published: July 31, 2024

Spiking neural networks (SNNs) offer a promising energy-efficient alternative to artificial (ANNs), in virtue of their high biological plausibility, rich spatial-temporal dynamics, and event-driven computation. The direct training algorithms based on the surrogate gradient method provide sufficient flexibility design novel SNN architectures explore dynamics SNNs. According previous studies, performance models is highly dependent sizes. Recently, deep SNNs have achieved great progress both neuromorphic datasets large-scale static datasets. Notably, transformer-based show comparable with ANN counterparts. In this paper, we new perspective summarize theories methods for systematic comprehensive way, including theory fundamentals, spiking neuron models, advanced residual architectures, software frameworks hardware, applications, future trends.

Language: Английский

---

Photonic Spiking Neural Networks with Highly Efficient Training Protocols for Ultrafast Neuromorphic Computing Systems

Intelligent Computing, Journal Year: 2023, Volume and Issue: 2

Published: Jan. 1, 2023

Photonic technologies offer great prospects for novel, ultrafast, energy-efficient, and hardware-friendly neuromorphic (brain-like) computing platforms. Moreover, photonic approaches based on ubiquitous, technology-mature, low-cost vertical-cavity surface-emitting lasers (VCSELs) (devices found in fiber-optic transmitters, mobile phones, automotive sensors) are of particular interest. Given that VCSELs have shown the ability to realize neuronal optical spiking responses (at ultrafast GHz rates), their use spike-based information-processing systems has been proposed. In this study, neural network (SNN) operation, a system just one VCSEL, is reported alongside novel binary weight “significance” training scheme fully capitalizes discrete nature spikes used by SNN process input information. The VCSEL-based was tested with highly complex multivariate classification task (MADELON) before its performance compared using traditional least-squares method an alternative weighting scheme. Excellent accuracies >94% were achieved both methods, exceeding benchmark dataset fraction processing time. newly also dramatically reduces set size requirements number trained nodes (≤1% total node count). This SNN, combination scheme, therefore grants reduced hardware complexity potential application future artificial intelligence applications.

Language: Английский

---

Simulation and implementation of two-layer oscillatory neural networks for image edge detection: bidirectional and feedforward architectures

Neuromorphic Computing and Engineering, Journal Year: 2023, Volume and Issue: 3(1), P. 014006 - 014006

Published: Jan. 13, 2023

Abstract The growing number of edge devices in everyday life generates a considerable amount data that current AI algorithms, like artificial neural networks, cannot handle inside with limited bandwidth, memory, and energy available. Neuromorphic computing, low-power oscillatory networks (ONNs), is an alternative attractive solution to solve complex problems at the edge. However, ONN currently its fully-connected recurrent architecture auto-associative memory problems. In this work, we use two-layer bidirectional architecture. We introduce feedforward perform image detection, using replace convolutional filters scan image. Using HNN Matlab emulator digital design simulations, report efficient detection from both architectures various size (3 × 3, 5 5, 7 7) on black white images. contrast, can also gray scale With design, assess latency performances obtain 3 filter real-time (camera flow 25 30 images per second) up 128 pixels while same deal 170 pixels, due faster computation.

Language: Английский

---

A Review of Computing with Spiking Neural Networks

Computers, materials & continua/Computers, materials & continua (Print), Journal Year: 2024, Volume and Issue: 78(3), P. 2909 - 2939

Published: Jan. 1, 2024

Artificial neural networks (ANNs) have led to landmark changes in many fields, but they still differ significantly from the mechanisms of real biological and face problems such as high computing costs, excessive power, so on.Spiking (SNNs) provide a new approach combined with brain-like science improve computational energy efficiency, architecture, credibility current deep learning applications.In early stage development, its poor performance hindered application SNNs real-world scenarios.In recent years, made great progress practicability compared earlier research results, are continuously producing significant results.Although there already pieces literature on SNNs, is lack comprehensive review perspective improving practicality well incorporating latest results.Starting this issue, paper elaborates along complete usage process including network construction, data processing, model training, deployment, aiming more practical guidance promote development SNNs.Therefore, connotation status SNN reviewed systematically comprehensively four aspects: composition structure, set, algorithm, software/hardware platform.Then characteristics intelligent summarized, challenges discussed future directions also prospected.Our shows that fields machine computing, comparable scale ANNs ability challenge large datasets variety tasks.The advantages over terms efficiency spatial-temporal processing been fully exploited.And programming deployment tools has lowered threshold for use SNNs.SNNs show broad prospect computing.

Language: Английский

---

From Brain Models to Robotic Embodied Cognition: How Does Biological Plausibility Inform Neuromorphic Systems?

Brain Sciences, Journal Year: 2023, Volume and Issue: 13(9), P. 1316 - 1316

Published: Sept. 13, 2023

We examine the challenging “marriage” between computational efficiency and biological plausibility—A crucial node in domain of spiking neural networks at intersection neuroscience, artificial intelligence, robotics. Through a transdisciplinary review, we retrace historical most recent constraining influences that these parallel fields have exerted on descriptive analysis brain, construction predictive brain models, ultimately, embodiment an enacted robotic agent. study models Spiking Neural Networks (SNN) as central means enabling autonomous intelligent behaviors systems. then provide critical comparison available hardware software to emulate SNNs for investigating entities their application Neuromorphics is identified promising tool embody real physical systems different neuromorphic chips are compared. The concepts required describing dissected contextualized new no man’s land cognitive neuroscience intelligence. Although there reviews computing various modules guidance, navigation, control systems, focus this paper more closing cognition loop SNN-embodied argue biologically viable neuronal used electroencephalogram signals excellent candidates furthering our knowledge explainability SNNs. complete survey by reviewing can benefit from hardware, e.g., perception (with vision), localization, cognition. conclude tradeoff symbolic power plausibility be best addressed neuromorphics, whose presence neurorobotics provides accountable empirical testbench synthetic natural embodied where both theoretical future work should converge multidisciplinary efforts involving

Language: Английский

---

Event-driven adaptive optical neural network

Science Advances, Journal Year: 2023, Volume and Issue: 9(42)

Published: Oct. 20, 2023

We present an adaptive optical neural network based on a large-scale event-driven architecture. In addition to changing the synaptic weights (synaptic plasticity), network's structure can also be reconfigured enabling various functionalities (structural plasticity). Key building blocks are wavelength-addressable artificial neurons with embedded phase-change materials that implement nonlinear activation functions and nonvolatile memory. Using multimode focusing, function features both excitatory inhibitory responses shows reversible switching contrast of 3.2 decibels. train distinguish between English German text samples via evolutionary algorithm. investigate structural plasticity during training process. On basis this concept, we realize consisting 736 subnetworks 16 material each. Overall, 8398 functional, highlighting scalability photonic

Language: Английский

---

Spiking two-stream methods with unsupervised STDP-based learning for action recognition

Signal Processing Image Communication, Journal Year: 2025, Volume and Issue: unknown, P. 117263 - 117263

Published: Jan. 1, 2025

Language: Английский

---

Scalable network emulation on analog neuromorphic hardware

Frontiers in Neuroscience, Journal Year: 2025, Volume and Issue: 18

Published: Feb. 5, 2025

We present a novel software feature for the BrainScaleS-2 accelerated neuromorphic platform that facilitates partitioned emulation of large-scale spiking neural networks. This approach is well suited deep networks and allows sequential model on undersized resources if largest recurrent subnetwork required neuron fan-in fit substrate. demonstrate training two network models—using MNIST EuroSAT datasets—that exceed physical size constraints single-chip system. The ability to emulate train larger than substrate provides pathway accurate performance evaluation in planned or scaled systems, ultimately advancing development understanding models computing architectures.

Language: Английский

---

Sequence Learning with Analog Neuromorphic Multi-Compartment Neurons and On-Chip Structural STDP

Lecture notes in computer science, Journal Year: 2025, Volume and Issue: unknown, P. 207 - 230

Published: Jan. 1, 2025

Language: Английский

---

Photonic spiking neural network built with a single VCSEL for high-speed time series prediction

Communications Physics, Journal Year: 2025, Volume and Issue: 8(1)

Published: March 20, 2025

Language: Английский

---