Sign language recognition using micro-Doppler and explainable deep learning

McCleary, James and Parra García, Laura and Ilioudis, Christos and Clemente, Carmine; (2021) Sign language recognition using micro-Doppler and explainable deep learning. In: 2021 IEEE Radar Conference (RadarConf21). IEEE Radar Conference . IEEE, USA. ISBN 9781728176093 (https://doi.org/10.1109/RadarConf2147009.2021.9455...)

Full text not available in this repository.Request a copy

Abstract

In this paper, Sign Language Recognition and classification of the micro-Doppler signatures of different British Sign Language (BSL) gestures is studied. A database of four different BSL hand gesture motions is presented in the form of micro-Doppler signals, recorded with a continuous waveform radar. For detecting the presence of the micro-Doppler signatures, joint time-frequency is applied by calculating their spectrograms. Each individual gesture is expected to contain unique spectral characteristics that are exploited in order to classify the gestures. A deep learning approach with transfer learning is studied and discussed for carrying out the classification task. Following this, a novel explainable AI algorithm is implemented to give the user visual feedback, in the form of colour highlights, for the most relevant features used to classify each signal.

ORCID iDs

McCleary, James, Parra García, Laura, Ilioudis, Christos ORCID logoORCID: https://orcid.org/0000-0002-7164-6461 and Clemente, Carmine ORCID logoORCID: https://orcid.org/0000-0002-6665-693X;
CORE (COnnecting REpositories)