Sign language recognition using spiking neural networks
Chaudhari, Pranav and Vicente-Sola, Alex and Basu, Amlan and Manna, Davide L. and Kirkland, Paul and Di Caterina, Gaetano (2024) Sign language recognition using spiking neural networks. Procedia Computer Science, 235. pp. 2674-2683. ISSN 1877-0509 (https://doi.org/10.1016/j.procs.2024.04.252)
Preview |
Text.
Filename: Chaudhari-etal-PCS-2024-Sign-language-recognition-using-spiking.pdf
Final Published Version License: Download (596kB)| Preview |
Abstract
In recent years, research in automatic Sign Language Recognition (SLR) has undergone significant progress, serving as a foundational base for developing applications that aim to promote the integration of deaf individuals into society. Most of this progress is owed to the recent developments in deep learning. However, the deployment of conventional Artificial Neural Networks (ANNs) can be hindered by their requirements in terms of computational power and energy consumption. Therefore, to improve the efficiency of current SLR systems, in this work, we propose the use of the increasingly popular Spiking Neural Networks (SNNs), which, on the one hand, provide more energy-efficient computations than conventional ANNs and, on the other hand, are able to process temporal sequences with simpler architectures thanks to their temporal dynamics. To evaluate our method, we utilize WLASL300, the 300-word (300 classes of signs) dataset fromWord-Level American Sign Language, and achieve an improvement in accuracy with the SNN (+2.70%) over the previous state-of-the-art, when working with energy-efficient spiking neurons. Furthermore, we construct a non-spiking version of the same network and evaluate it in a similar manner. Our results demonstrate how the SNN has sparser activations (25% less), thanks to the use of spiking neurons, and therefore can be implemented with a lower power requirement than an ANN version of the same architecture. This work thus demonstrates the possibility of performing SLR in a very effective and efficient way, thus opening up the development of applications that span from the automatic real-time translation of dynamic signs to remote control utilizing sign languages.
-
-
Item type: Article ID code: 86830 Dates: DateEvent31 May 2024Published25 September 2023AcceptedSubjects: Technology > Electrical engineering. Electronics Nuclear engineering
Science > Mathematics > Electronic computers. Computer science > Other topics, A-ZDepartment: Faculty of Engineering > Electronic and Electrical Engineering Depositing user: Pure Administrator Date deposited: 03 Oct 2023 10:34 Last modified: 05 Aug 2024 01:40 Related URLs: URI: https://strathprints.strath.ac.uk/id/eprint/86830