A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition

Xing, Yannan and Di Caterina, Gaetano and Soraghan, John (2020) A new spiking convolutional recurrent neural network (SCRNN) with applications to event-based hand gesture recognition. Frontiers in Neuroscience, 14. 590164. ISSN 1662-453X

[img]
Preview
Text (Xing-etal-FN-2020-A-new-spiking-convolutional-recurrent-neural-network)
Xing_etal_FN_2020_A_new_spiking_convolutional_recurrent_neural_network.pdf
Final Published Version
License: Creative Commons Attribution 4.0 logo

Download (2MB)| Preview

    Abstract

    The combination of neuromorphic visual sensors and spiking neural network offer a high efficient bio-inspired solution to real-world applications. However, processing event- based sequences still remains challenging because of the nature of their asynchronism and sparsity behaviour. In this paper, a novel spiking convolutional recurrent neural network (SCRNN) architecture that takes advantage of both convolution operation and recurrent connectivity to maintain the spatial and temporal relations from event-based sequence data is presented. The use of recurrent architecture enables the network to have arbitrary length of sampling window allowing the network to exploit temporal correlations between event collections. Rather than standard ANN to SNN conversion techniques, the network utilizes supervised Spike Layer Error Reassignment(SLAYER) training mechanism that allows the network to directly adapt to neuromorphic(event-based) data. The network structure is validated on the DVS gesture dataset and it has achieved a 10 class gesture recognition accuracy of 96.59% and 11 class gesture recognition accuracy of 92.01%.