Frontiers in Computational Neuroscience, Journal Year: 2024, Volume and Issue: 18

Published: Nov. 26, 2024

Event-based cameras are suitable for human action recognition (HAR) by providing movement perception with highly dynamic range, high temporal resolution, power efficiency and low latency. Spike Neural Networks (SNNs) naturally suited to deal the asynchronous sparse data from event due their spike-based event-driven paradigm, less consumption compared artificial neural networks. In this paper, we propose two end-to-end SNNs, namely Spike-HAR Spike-HAR++, introduce spiking transformer into event-based HAR. includes novel blocks: a spike attention branch, which enables model focus on regions rates, reducing impact of noise improve accuracy, parallel block simplified self-attention mechanism, increasing computational efficiency. To better extract crucial information high-level features, modify architecture branch extend it in higher dimension, proposing Spike-HAR++ further enhance classification performance. Comprehensive experiments were conducted four HAR datasets: SL-Animals-DVS, N-LSA64, DVS128 Gesture DailyAction-DVS, demonstrate superior performance our proposed model. Additionally, require only 0.03 0.06 mJ, respectively, process sequence frames, sizes 0.7 1.8 M. This positions as promising new SNN baseline community. Code is available at Spike-HAR++.

Language: Английский