A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition

Tools

Xing, Yannan and Di Caterina, Gaetano and Soraghan, John (2020) A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition. Frontiers in Neuroscience, 14. 590164. ISSN 1662-453X (https://doi.org/10.3389/fnins.2020.590164)

[thumbnail of Xing-etal-FN-2020-A-new-spiking-convolutional-recurrent-neural-network]
Preview
 Text. Filename: Xing_etal_FN_2020_A_new_spiking_convolutional_recurrent_neural_network.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo
Download (2MB)| Preview

Abstract

The combination of neuromorphic visual sensors and spiking neural network offers a high efficient bio-inspired solution to real-world applications. However, processing event- based sequences remains challenging because of the nature of their asynchronism and sparsity behavior. In this paper, a novel spiking convolutional recurrent neural network (SCRNN) architecture that takes advantage of both convolution operation and recurrent connectivity to maintain the spatial and temporal relations from event-based sequence data are presented. The use of recurrent architecture enables the network to have a sampling window with an arbitrary length, allowing the network to exploit temporal correlations between event collections. Rather than standard ANN to SNN conversion techniques, the network utilizes a supervised Spike Layer Error Reassignment (SLAYER) training mechanism that allows the network to adapt to neuromorphic (event-based) data directly. The network structure is validated on the DVS gesture dataset and achieves a 10 class gesture recognition accuracy of 96.59% and an 11 class gesture recognition accuracy of 90.28%.

ORCID iDs

Xing, Yannan, Di Caterina, Gaetano ORCID logoORCID: https://orcid.org/0000-0002-7256-0897 and Soraghan, John ORCID logoORCID: https://orcid.org/0000-0003-4418-7391;
CORE (COnnecting REpositories)